Thames path improvements – Letter to Oxfordshire County Council

Tim Pizey from Tim Pizey

I have sent the following letter to LTS.Team AT in response to




I have submitted the following via the questionnaire:


The Oxpens bridge should move upstream to parallel the railway bridge with a path extension besides railway to station. Getting from Thames/railway station to canal towpath and up to Kidlington Airport and Kidlington new housing extension needed. Second bridge, again parallel to rail bridge needed to access Science Park.


I have used the Thames towpath for commuting to Osney Mead industrial estate for three months and for work on Cornmarket for three years. I also used the path for three months to commute to Kidlington (Oxford Airport) but it is too arduous and much to my sadness I now cycle on the road, so have trenchant views on the difficulty of cycle commuting in Oxford.


I understand that there is a plan to build many new houses in the Kidlington gap, between Summertown and Kidlington.


These new commuters would very much appreciate a functioning cycle route into the centre of town.


The new Oxford Parkway should be integrated into this cycle route.


The problem facing cyclists is not how to get from the towpath to the Centre, this is served by the pipe bridge, the pedestrian crossing down stream and Folley Bridge. What is needed is easy access to the railway station which could be easily achieved by a bridge parallel to the railway bridge and then along railway land to the station itself.


Cyclists wishing to get from the Thames to the canal have to continue to Osney, cross the road and then fiddle around the Thames onto the canal path, which is in a very poor state, then up to Kingsbridge and all the way through Kidlington to the Oxford Airport.


Finally the Thames path should connect the Science Park, and the planned new housing at Littlemore. This again could be achieved by a cycle bridge parallel to the existing railway bridge down stream from the bypass underpass.


I really welcome the proposals but would urge you to consider extending its scope and vision. This could be such a good route and would show that Oxford is a cycling city.


best regards Tim Pizey

-- Tim Pizey -

Tell, don’t ask

Tim Pizey from Tim Pizey

More than twelve years ago Tim Joyce passed on some programming wisdom:

With programs tell don't ask, vice versa for people.

This was a bit abstract for me at the time but last night it came back to me as what is wrong with the code I am currently working on. We store our application configuration in a table in the system's target database and whenever some configuration is needed it is looked up in the database. There was no problem with this approach when the code was written because JUnit had not been invented and testing was not the main part of our discipline. However to write a test we would need a database present, which is an obstacle to fast, distinct, unit tests and has been a blocker to writing tests.

Noncompliant Code Example

public class Example { 
private String path;
public void logPath() {
try {
path = CachedSystemParameter.getInstance().
} catch (SystemParameterException e) {
logger.error("[BUSINESS] Error while retrieving system parameter PATH", e);
}"Path: " + path);

Compliant Code Example

By adding sftpPath to the class constructor we can test the business logic without the need for a database fixture.
public class Example { 

private String path;

public Example() {

public Example(String path) {
this.path = path;

public void logPath() {"Path: " + path);

Testing java slf4j over log4j logging in JUnit using SLF4J Test

Tim Pizey from Tim Pizey

Testing logging on failure paths has two problems:

  • It is hard to get the log message text
  • The logger outputs to the test log
The first leads to compromises eg verifying only that a message was logged, the second makes you, the programmer, think an error has occurred when the tests in fact passed.

Code to test

public class Sut {
public String perform() {
getLog().debug("In perform");
return "Hello world";

My clunky PowerMock Solution

My approach was problematic as it required the use of PowerMock which is as powerful as nitroglycerin.

Test Code

public class SutTest {

public void testPerform() {
Logger mockLog = mock(Logger.class);

assertEquals("Hello world", new Sut().perform());
verify(mockLog, times(1)).debug(startsWith("In perform"));

Elegant SLF4j Test Solution

The slf4j-test project by RobElliot266 provides a logger which stores messages and so can be asserted against.

POM Setup

Add the following to your dependencies


To ensure that this logger is used during tests only and that it takes precedence over the production logger in the test class path ensure the test logger is the first logger mentioned in the dependencies block and has a test scope.

As an additional measure you can explicitly exclude the production logger from the test class path:


Test Code

public class SutTest {
public void testPerform() {
assertEquals("Hello world", new Sut().perform());
assertEquals("Testing", logger.getLoggingEvents().get(0).getMessage());

Much thanks to RobElliot266 for an neat solution to a problem that has been bugging me for a while.

Migrate MelatiSite from CVS to github

Tim Pizey from Tim Pizey

Re-visiting (why did I not complete this at the time?)

Following How to export revision history from mercurial or git to cvs?

On hanuman I created an id file git_authors mapping cvs ids to github name, email format for all contributors:

timp=Tim Pizey<>
then create a repository on github (melati in this example, I already have uploaded my ssh public key for this machine)

cd ~
git cvsimport -d /usr/cvsroot -C MelatiSite -r cvs -k -A git_authors MelatiSite

cd melati
echo A jdbc to java object relational mapping system. 1999-2011 > README.txt
git add README.txt
git commit -m "Initial" README.txt
git remote add origin
git push -u origin master

CentOS setup on VirtualBox

Tim Pizey from Tim Pizey

Once you have Networking working there is still a long way to go.

yum groupinstall "Development Tools"
yum install kernel-devel
yum install kde-workspace
yum group install "X Window System"
yum groupinstall "Fonts"
yum install gdm

Now we can login without a GUI but startx when one is needed.

Installing Guest Additions

The guest Centos is a stock distribution, you have to tell it that it is inside VirtualBox.

Make the additions visible to the guest:

In the "Devices" menu in the virtual machine's menu bar, VirtualBox has a handy menu item named "Insert Guest Additions CD image", which mounts the Guest Additions ISO file inside your virtual machine.

yum install dkms
mkdir -p /media/cdrom
# Note change from /dev/scd0 in CentOS6
mount /dev/sr0 /media/cdrom
sh /media/cdrom/

We are now able to move the mouse seamlessly between our guest and host and window systems understand each other.

Sharing files between the host and guest

In the host (Windows) create C:\vbshared and using the VirtualBox interface share this with the guest. In the guest:

mkdir /vbshared
mount -t vboxsf vbshared /vbshared

it will be visible as /vbshared/ from inside the guest.

Current Software Development Pre-Requisites

Tim Pizey from Tim Pizey

When starting a new project or joining an existing one there are a number of tools and features which should be in place. I have ordered them in order both of importance and the order in which the global community learnt the painful lessons that none of these are optional.

This is based upon Project initiation - a recipe.

Short name

Google it, ensure it is available as a url, check twitter.


If there is no README create it now!

Source control

The only decision is public or private. It will be a git repo.

If any other SCM system is in place convert to git before doing anything else.

Decide on git usage strategy: git flow, release branches, developer forks with feature branches and merge to master.

Development machine

Do we really want to develop in Fortran under VMS? oh, OK.

Develop on the operating system you are deploying to. If you develop on OSX and deploy to debian it will bite you. Developing for Redhat using Windows should be made illegal.

Continuous Integration

Jenkins of course.

Track the code coverage, anything less than 100&percent; is not acceptable.

Static Analysis

For legacy projects Sonar establishes a baseline, for new projects it holds the line throughout the projects life.

Continuous Deployment

The closer to Continuous Deployment the fewer platform types are needed.


Metrics enable blue green deployment and A/B testing.

Issue tracking and work planning

Just you: gitthub, team: Jira

Continuous Availability for a Jenkins Continuous Integration Service

Tim Pizey from Tim Pizey

When your CI server is becoming too big to fail

This post was written when I was responsible for a heavily used CI server, for a company which is no longer trading, so the tenses may be a mixed

Once an organisation starts to use Jenkins, and starts to buy into the Continuous Integration methodology, very quickly the Continuous Integration server becomes indispensable.

The Problem

The success of Jenkins is based upon its plugin based architecture. This has enabled Kohsuke Kawaguchi to keep tight control over the core whilst allowing others to contribute plugins. This has led to rapid growth of the community and a very low bar to contributing (there are currently over 1000 plugins).

Each plugin has the ability to bring your CI server to a halt. Whilst there is a Long Term Support version of Jenkins the plugins, which supply almost all of the functionality, do not have any enforced gate keeping.

Solution Elements

A completely resilient CI service is an expensive thing to achieve. The following elements must be applied baring in mind the proportion of the risk of failure they mitigate.

Split its jobs onto multiple CI servers

Use of personal Jenkins installations is recommended, but there is still a requirement for a single, central server.

This should be a last resort, splitting tasks out across slaves achieves many of the benefits without losing a single reporting point.

Split jobs out to SSH slaves
We had a misconfiguration of our ssh slaves such that they install the Jenkins package. The only use of the package is to ensure that the jenkins user is present, though tasks should not, ideally, be run as the jenkins user.

One disadvantage of using ssh slaves is that it requires copies of the ssh keys to be manually copied from the master server to the slaves.

Because jobs are initiated from master to the slave the master cannot be restarted during a job's execution (this is currently also true for JNLP slaves, but is not necessarily so).

The main disadvantage of ssh slaves is that by referencing real slaves they make the task of creating a staging server more complex, as a simple copy of the master would initiate jobs on the real slaves.

Split jobs out to JNLP slaves

Existing ssh slave jobs should be left unchanged until they can be replaced. This is a blocker on creating a staging CI server.

This is the recommended setup, which we used eventually for most jobs.

Minimise Shared Resources

Most of these problems can be overcome by spinning up a virtual machine for each job, from scratch, provisioned by puppet via vagrant.

In addition to sharing plugins, and hence sharing faulty plugins, another way in which jobs can adversely interact is by their use of shared resources(disk space, memory, cpus) and shared services(databases, message queues, mail servers, web application servers, caches and indexes).

Run the LTS version on production CI servers

Move to LTS at the earliest opportunity.

There are two plugin feeds, one for bleeding edge, the other for LTS.

Strategies for Plugin upgrade

Hope and trust

Up until our recent problem I would have said that the Jenkins community is pretty high quality, most plugins do not break your server, your ability to predict which ones will break your installation is small so brace yourself and be ready to fix and report any problems that there are. I have run three servers for five years and not previously had a problem.

Upgrade plugins one at a time, restart server between each one.

This seems reasonable, but at a release rate of 4.3 per day, seven days a week since 2011-02-21 even your subset of plugins are going to get updated quite frequently.

Use a staging CI server, if you can

If your CI server and its slaves are all setup using puppet, then you can clone it all, including repositories and services, so that any publishing acts do not have any impact on the real world, otherwise you will send emails and publish artefacts which interfere with your live system. Whilst we are using ssh slaves the staging server would either initiate jobs on real slaves or they too would need to be staged.

Use a partial staging CI server
Jobs which publish an artefact every time they are run cannot be re-run so are not suitable for running on a staging server.

You can prune your jobs down to those which are idempotent, ie those which do not publish and do not use ssh slaves, but the non-idempotent jobs cannot be re-run.

Control and monitor the addition of plugins

Users intending to install a plugin should ask on irc, giving the plugin url.

From the above it is clear that for a production CI server the addition of plugins is not risk or cost free.

Remove unused plugins, after consulting original installer

We still have a number of redundant plugins installed.

Plugins build up over time.

Monitor the logs

Currently there is no monitoring of the Jenkins log.

A log monitor which detects java exceptions might be used.

Backup the whole machine

Whilst the machine is backed up a fire drill is needed to prove that a state can be returned to.

Once a month restore from backup to a clean machine.

Store the configuration in Git

The configuration of Jenkins has been stored, and restored from.

This process is only one element of recreating a server. Once a month restore from git to a clean machine.