Git remote repos with OneDrive

Pete Barber from C#, C++, Windows & other ramblings

I have various public git repositories on GitHub but I like to keep some source (usually my active App Store apps) private. Whilst it'd be nice to use GitHub private repositories, given my Apps are for fun and don't really make anything, don't require collaboration then the pricing is prohibitive.

However, I really like the idea of having an offsite copy of my repository. As it happens I have an Office 365 Subscription which comes with 1TB of OneDrive space. I use OneDrive on OSX to sync a bunch of folders. I could use a synchronised OneDrive for my work directory but I don't want OneDrive synchronising all the temporary build files etc every time I build.

It turns out the ideal solution is to create a remote clone in a OneDrive synchronised folder. In fact I have a dedicated OneDrive folder called 'src' that contains clones of all of my git repos. Then, each time I commit to the local git repository and push OneDrive performs the synchronisation. If it happens that the code I'm working on is public then adding a public GitHub repo is a doddle.

Having created a local Git repo (though usually Xcode does this when starting a new project) it's easy to create the OneDrive clone:

  1. cd /Users/Pete/OneDrive/src
  2. git clone --bare file:////Users/Pete/Projects/<ProjectName>/.git <ProjectName>.git
As the remote clone is really just a backup come master that will never be working repository I create a bare clone and suffix the directory name with '.git'. I think this is a fairly common convention.

At this point the source repository is cloned but the source is now a remote of the new repository rather than than the other way round. This is easy to fix:

cd-ing into <ProjectName>.git (my example Project is Photone> and running git remotes gives:

~/OneDrive/src/Photone.git[23]git remote -v
origin file:////Users/Pete/Projects/PhotoneViewer/.git (fetch)
origin file:////Users/Pete/Projects/PhotoneViewer/.git (push)

than running:
  1. git remote remove origin
Removes the relationship between the new remote and the original source. To implement the desired reverse relationship:


  1. cd /Users/Pete/Projects<ProjectName>
  2. git remote add OneDrive file:////Users/Pete/OneDrive/src/<ProjectName>.git
I name my OneDrive remote repos 'OneDrive'. This helps if I have multiple remotes.

git remote (for my current project) now gives:


~/Projects/PhotoneViewer[49]git remote -v
OneDrive file://Users/Pete/OneDrive/src/Photone.git (fetch)
OneDrive file://Users/Pete/OneDrive/src/Photone.git (push)


From this point onwards I use SourceTree. However, if you use the command line then a couple of extra steps are required otherwise git complains. 

Firstly, when pushing if you just want to do:

git push OneDrive you need to tell git that the new OneDrive remote is the master. This is done by:

git push --set-upstream OneDrive master

Secondly, unless you've already set the push.default setting or just use 'git push --all' then you'll need to decide which option you want. The help from git describes these well:

Git 2.0 from 'matching' to 'simple'. To squelch this message
and maintain the traditional behavior, use:

  git config --global push.default matching

To squelch this message and adopt the new behavior now, use:

  git config --global push.default simple

When push.default is set to 'matching', git will push local branches
to the remote branches that already exist with the same name.

Since Git 2.0, Git defaults to the more conservative 'simple'
behavior, which only pushes the current branch to the corresponding
remote branch that 'git pull' uses to update the current branch.

See 'git help config' and search for 'push.default' for further information.
(the 'simple' mode was introduced in Git 1.7.11. Use the similar mode
'current' instead of 'simple' if you sometimes use older versions of Git)

If you're using the remote clone as a backup then perhaps the original, i.e. matching behaviour is desirable. Whilst I use OneDrive this configuration should work for any other file synchronisation service, e.g. iCloud, DropBox or a standard mounted File System, e.g. Samba, NFS etc.

Swift Enums and Protocols

Pete Barber from C#, C++, Windows &amp; other ramblings

I'm trying to clear out my inbox before Christmas and I noticed an emails to myself entitled 'Enum question. Add protocol to enum?'.

The short answer is yes. The longer one. Take the following protocol

protocol Foo
{
func f() -> Void
}

A simple enum can be created that implements it:

enum Test: Foo
{
case One
func f()
{
print("Hello")
}

}

So the following short program:

let baz = Test.One

baz.f()

generates the output:

Hello
Program ended with exit code: 0

The protocol can also be implemented by extension so the following is equivalent and produces the same results:

extension Test: Foo
{
func f()
{
print("Hello")
}
}

It's not surprising that Swift's enum types support protocols. Off the top of my head I can't think of any clever reasons why you'd have enums implement a protocol. Given an enum is a first class type in Swift then it makes perfect sense. In fact it's documented on page 424 in The Swift Programming Language.

Continuous Deployment of a platform and its variants using githook and cron

Tim Pizey from Tim Pizey

Architecture

We have a prototypical webapp platform which has four variants, commodities, energy, minerals and wood. We use the Maven war overlay feature. Don't blame me it was before my time.

This architecture, with a core platform and four variants, means that one commit to platform can result in five staging sites needing to be redeployed.

Continuous Integration

We have a fairly mature Continuous Integration setup, using Jenkins to build five projects on commit. The team is small enough that we also build each developer's fork. Broken builds on trunk are not common.

NB This setup does deploy broken builds. Use a pipeline if broken staging builds are a problem in themselves.

Of Martin Fowler's Continuous Integration Checklist we have a score in every category but one:

  • Maintain a Single Source Repository
    Bitbucket.org
  • Automate the Build
    We build using Maven.
  • Make Your Build Self-Testing
    Maven runs our Spring tests and unit tests (coverage could be higher).
  • Everyone Commits To the Mainline Every Day
    I do, some with better memories keep longer running branches.
  • Every Commit Should Build the Mainline on an Integration Machine
    Jenkins as a service from Cloudbees.
  • Fix Broken Builds Immediately
    We have a prominently displayed build wall, the approach here deploys broken builds to staging so we rely upon having none.
  • Keep the Build Fast
    We have reduced the local build to eight minutes, twenty five minutes or so on Jenkins. This is not acceptable and does cause problems, such as tests not being run locally, but increasing the coverage and reducing the run time will not be easy.
  • Test in a Clone of the Production Environment
    There is no difference in kind between the production and development environments. Developers use the same operating system and deployment mechanism as is used on the servers.
  • Make it Easy for Anyone to Get the Latest Executable
    Jenkins deploys to a Maven snapshot repository.
  • Everyone can see what's happening
    Our Jenkins build wall is on display in the coding room.
  • Automate Deployment
    The missing piece, covered in this post.

Continuous, Unchecked, Deployment

Each project has an executable build file redo:


mvn clean install -DskipTests=true
sudo ./reload
which calls the deployment to tomcat reload

service tomcat7 stop
rm -rf /var/lib/tomcat7/webapps/ROOT
cp target/ROOT.war /var/lib/tomcat7/webapps/
service tomcat7 start
We can use a githook to call redo when the project is updated:
cd .git/hooks
ln -s ../../redo post-merge
Add a line to chrontab using
crontab -e

*/5 * * * * cd checkout/commodities && git pull -q origin master
This polls for changes to the project code every five minutes and calls redo if a change is detected.

However we also need to redeploy if a change is made to the prototype, platform.

We can achieve this with a script continuousDeployment


#!/bin/bash

# Assumed to be invoked from derivative project which contains script redo

pwd=`pwd`
cd ../platform
git fetch > change_log.txt 2>&1
if [ -s change_log.txt ]
then
# Installs by fireing githook post-merge
git pull origin master
cd $pwd
./redo
fi
rm change_log.txt
which is also invoked by a crontab:

*/5 * * * * cd checkout/commodities && ../platform/continuousDeployment

PC-lint Plus is coming – and with it, full support for C++ 11 and C++ 14

Products, the Universe and Everything from Products, the Universe and Everything

Gimpel have just announced a beta of PC-lint Plus - a new product which will sit alongside PC-lint. To understand the (rather complex, I'm afraid) background and the background to why a beta of PC-lint Plus has been announced rather than one for the long awaited 9.00m patch, please keep reading... It is no secret that PC-lint 9.0 struggles with some C++ 11/14 code - and especially any code making use of C++ 11 variadic templates. The result is errors - and nasty, hard to work around ones at that. Under Windows, this is particularly noticable when analysing projects written for Visual Studio 2013 and 2015, which both use variadic templates in their system headers (Visual Studio 2012 and earlier did not). A glance at the C++ 11 support page on the Gimpel website illustrates this rather well, with the following C++ 11 features shown as not yet being supported as of PC-lint 9.00L:
  • Allowing move constructors to throw [noexcept] (N3050)
  • Variadic templates (N2242)
  • Extending variadic template template parameters (N2555)
  • Initializer lists (N2672)
  • Generalized attributes (N2761)
  • Generalized constant expressions (N2235)
  • Inheriting constructors(N2540)
  • Unicode string literals (N2442)
  • Raw string literals (N2442)
  • Universal character names in literals (N2170)
  • User-defined literals (N2765)
  • Inline namespaces (N2535)
  • Unrestricted unions (N2544)
  • Minimal support for garbage collection and reachability-based leak detection (N2670)
  • Extended integral types (N1988)
That's just of course from C++ 11, and with C++ 14 also out and C++ 17 (which like C++ 11 promises major changes to the language) is just around the corner, keeping up has been getting more and more difficult. As Jim Gimpel said to us recently:
Unfortunately, keeping up with the evolving C++ standards which seemed to know no bounds, became more and more difficult as Gimpel Software did have bounds.
It is worth remembering that PC-lint was first introduced in 1985, so presumably the C++ front end in 9.00L is a direct descendent of that in the original product rather than one of the standard C++ front ends (EDG, Clang, GCC etc.) our industry has effectively settled upon since. As a result, every change in the language no doubt requires comparable changes in the PC-lint front end. Given how much C++ has evolved recently (and continues to evolve, looking forward to C++ 17 and beyond), that's potentially a huge ongoing task. Like many others, for some time we have been asking Gimpel for an estimate of roughly when an update addressing the remaining C++ 11 issues were realistically likely to become available. Although throughout last year we thought that support for variadic templates issue (the missing feature which seemed to be catching most people out) would be resolved with 9.00L in the Autumn sadly it was not to be, and by the start of this year it became obvious to us that we would have to put into place a workaround to allow customers to analyse Visual Studio 2013 and 2015 projects. Hence the system headers compatibility thunk described in Visual Studio 2013, PC-lint and C++ 11 Variadic Templates and available in Visual Lint 4.5.9.239 onwards. Around the same time, Gimpel also updated us on their future plans, and how they intended to offer complete C++ 11/14 support. Although we couldn't talk about that publicly at the time, now that they have announced the beta we are free to do so. The bottom line is that rather than continue to try to keep the existing PC-lint front end up to date with changes in the C++ Standard, Gimpel have decided to develop a spin-off product using the industry standard Clang front-end. The existing PC-lint 9.0 and Flexelint 9.0 products will continue to be supported for the foreseeable future. This is of course a major development, and in consequence is one that couldn't have been released as a PC-lint 9.0 patch anyway. As the Clang source requires a modern C++ compiler to compile it, this makes distributing PC-lint Plus as obfuscated source (as Flexelint currently is) impractical as many compilers for non-x86 platforms would not be able to compile it. The move to Clang also means that in future PC-lint Plus will be able to keep up with changes in the C++ standard as fast as new versions of Clang can be integrated. So, if (for example) the C++ Module System proposals (N4465/N4466) (which allow imports of compiled units to replace #includes of header files, thus drastically reducing build times) make it into C++ 17, PC-lint will be able to keep up without another major hiatus. For Windows, both 32 bit (pclp32.exe) and 64 bit (pclp64.exe) binaries will be provided along with full support for C++ 11/14, Visual Studio 2013 and 2015. For Linux, a pre-built x86 binary will be available which should simplify things for developers on that platform (and potentially allow the pricing of the Linux version of PC-lint to become more competitive with the Windows variant). For non-x86 platforms etc., Flexelint will continue to be available in its current form. The new product supports a considerable number of new messages (far too many to list!). In addition, obsolete options such as +linebuf and +macros have been removed. As far as our own products are concerned, Visual Lint 5.0.6.254 onwards are aware of PC-lint Plus (we've actually been testing with alpha builds since the summer) and should work with beta versions. Support for 64 bit builds of PC-lint will become available in the New Year, and now that the beta has been announced we will be adding documentation for the new messages to the message database installed with the product as information becomes available. Overall this is a huge step forward, but we are aware that it may come as a shock to many who were expecting PC-lint 9.00m. As soon as we know the details of Gimpel's PC-lint 9.0 to PC-lint Plus upgrade policy rest assured we will be talking about it. In the meantime, please do join the beta programme and help test it! Gimpel's beta announcement can be found in the blogpost Upcoming Support for C++14/VS2015.