Evidence-based SE groups doing interesting work, 2021 version

Derek Jones from The Shape of Code

Who are the research groups currently doing interesting work in evidenced-base software engineering (academics often use the term empirical software engineering)? Interestingness is very subjective, in my case it is based on whether I think the work looks like it might contribute something towards software engineering practices (rather than measuring something to get a paper published or fulfil a requirement for an MSc or PhD). I last addressed this question in 2013, and things have changed a lot since then.

This post focuses on groups (i.e., multiple active researchers), and by “currently doing” I’m looking for multiple papers published per year in the last few years.

As regular readers will know, I think that clueless button pushing (a.k.a. machine learning) in software engineering is mostly fake research. I tend to ignore groups that are heavily clueless button pushing oriented.

Like software development groups, research groups come and go, with a few persisting for many years. People change jobs, move into management, start companies based on their research, new productive people appear, and there is the perennial issue of funding. A year from now, any of the following groups may be disbanded or moved on to other research areas.

Some researchers leave a group to set up their own group (even moving continents), and I know that many people in the 2013 survey have done this (many in the Microsoft group listed in 2013 are now scattered across the country). Most academic research is done by students studying for a PhD, and the money needed to pay for these students comes from research grants. Some researchers are willing to spend their time applying for grants to build a group (on average, around 40% of a group’s lead researcher’s time is spent applying for grants), while others are happy to operate on a smaller scale.

Evidence-based research has become mainstream in software engineering, but this is not to say that the findings or data have any use outside of getting a paper published. A popular tactic employed by PhD students appears to be to look for what they consider to be an interesting pattern in code appearing on Github, and write a thesis that associated this pattern with an issue thought to be of general interest, e.g., predicting estimates/faults/maintainability/etc. Every now and again, a gold nugget turns up in the stream of fake research.

Data is being made available via personal Github pages, figshare, osf, Zenondo, and project or personal University (generally not a good idea, because the pages often go away when the researcher leaves). There is no current systematic attempt to catalogue the data.

There has been a huge increase in papers coming out of Brazil, and Brazilians working in research groups around the world, since 2013. No major Brazilian name springs to mind, but that may be because I have not noticed that they are Brazilian (every major research group seems to have one, and many of the minor ones as well). I may have failed to list a group because their group page is years out of date, which may be COVID related, bureaucracy, or they are no longer active.

The China list is incomplete. There are Chinese research groups whose group page is hosted on Github, and I have failed to remember that they are based in China. Also, Chinese pages can appear inactive for a year or two, and then suddenly be updated with lots of recent information. I have not attempted to keep track of Chinese research groups.

Organized by country, groups include (when there is no group page available, I have used the principle’s page, and when that is not available I have used a group member page; some groups make no attempt to help others find out about their work):

Belgium (I cite the researchers with links to pdfs)

Brazil (Garcia, Steinmacher)

Canada (Antoniol, Data-driven Analysis of Software Lab, Godfrey and Ptidel, Robillard, SAIL; three were listed in 2013)

China (Lin Chen, Lu Zhang)

Germany (Chair of Software Engineering, CSE working group, Software Engineering for Distributed Systems Group, Research group Zeller)

Greece (listed in 2013)


Italy (listed in 2013)

Japan (Inoue lab, Kamei Web, Kula, and Kusumoto lab)


Spain (the only member of the group listed in 2013 with a usable web page)

Sweden (Chalmers, KTH {Baudry and Monperrus, with no group page})

Switzerland (SCG and REVEAL; both listed in 2013)


USA (Devanbu, Foster, Maletic, Microsoft, PLUM lab, SEMERU, squaresLab, Weimer; two were listed in 2013)

Sitting here typing away, I have probably missed out some obvious candidates (particularly in the US). Suggestions for omissions welcome (remember, this is about groups, not individuals).

Visual Lint adds support for Visual Studio 2022

Products, the Universe and Everything from Products, the Universe and Everything

Visual Lint has now been released. This a recommended maintenance update for Visual Lint 8.0, and adds support for Visual Studio 2022 Preview:

Visual Lint running within Visual Studio 2022 Preview 5.0Visual Lint running within Visual Studio 2022 Preview 5.0

The following changes are included:

  • The Visual Studio plugin is now compatible with Visual Studio 2022 Preview.

  • The Visual Lint installer now includes dedicated VSIX extensions for Visual Studio 2017 (VisualLintPlugIn_vs2017.vsix), Visual Studio 2019 (VisualLintPlugIn_vs2019.vsix) and Visual Studio 2022 (VisualLintPlugIn_vs2022.vsix).

  • If makefile output is read while loading a C/C++ project, it is now used to determine which files will be built.

  • The Project Properties Dialog now includes a dedicated field for built-in (i.e. compiler defined) preprocessor definitions. Note that these include those inferred by other project settings, e.g. the Visual C++ "Runtime Library" property (compiler switches /MD[d], /MLd and /MT[d]) which infer _DEBUG/NDEBUG, _MT and _DLL.

  • Built-in (i.e. compiler defined) preprocessor symbols are now written separately in generated PC-lint/PC-lint Plus project indirect files from those specified directly.

  • Updated the PC-lint Plus compiler indirect file co-rb-vs2022.lnt to support Visual Studio 2022 v17.0.0 Preview 4.1.

  • Updated the PC-lint Plus compiler indirect file co-rb-vs2019.lnt to support Visual Studio 2019 v16.11.4.

  • Updated the VisualLintConsole help screen to reflect support for Visual Studio 2022 project and solution files.

  • Updated the installer to clarify that the "Remove Visual Lint commands from Visual Studio" option only applies to Visual Studio 2002-2008.

Download Visual Lint

How I prioritise planning over plans

Allan Kelly from Allan Kelly, Software Strategy

You might think that I’m a really organized person. After all, I spend a good chunk of my life helping other people be more organized about their work – and not just organized, prioritised, effective, and all those other good things. That might be true, people who know me well say I’m really well organized. But I always feel I’m faking it. I’m really disorganized.

As I spend a lot of time working by myself, for myself, and interleaving clients I need to organize my days. Over the years I’ve tried man different ways of organizing myself. Todo lists in my notebook are the main mechanism. Notebook and todo-list works well for the medium range but for the actual work of today its not so effective. I have, and sometimes use, a whiteboard: write out a list of things todo today and tick them as I do them. I’ve use post-it notes: write out all the things I need to do on one post-it each, prioritise them down the side of my desk and tick them as I do them.

In general I find that a system works for a while, maybe even a few of weeks but it decays. Perhaps its too routine, perhaps I’m over familiar. After a while I need a change. So after a period in the doldrums I bring back an old system or invent a new one.

During the last year of house-arrest I’ve found organizing myself really hard. A few months ago I came up with a new system: good old index cards. Extra big ones. On the left are the mundane or household things I need to do. On the right the important business stuff todo.

The keys to making any one of my systems work are:

1. The power of the (big red) tick. Being able to tick things off and mark them as done. Perhaps thats why electronic systems never work for me.

2. Prioritisation: Recognising that some things are more important and need to be done sooner or require more time. Accept some things fall off the bottom and don’t get done.

3. Limiting WIP, work in progress. It is easy to put too many things down, not do them, and write them down again tomorrow.

4. Sticking to the list and not getting distracted.

5. Rewrite the list every day

Now the last three there: prioritisation, limiting WIP and not getting distracted require personal discipline. I have to force myself to work within my system. Sometimes that is hard but if I don’t do it the system breaks down. And actually, that is usually why the system periodically requires reinventing.

For example: many of my cards list “EM, SL and LN” – short for E-Mail, Slack and LinkedIn. Messages arrive for me on all three and there are conversations I sometimes want to join in. But, very little on any of them is so important that is needs to be looked at at 9am. Everything on Slack and LinkedIn can wait, and almost everything on e-mail can wait. So a quick e-mail triage at 9am and pushing the rest until later in the day allows focus on important stuff. Unfortunately, because EM, SL and LN all generate dopamine it it very difficult to prevent myself from being distracted by them.

Rewriting the list everyday helps focus because I’m saying “this is what I will do.” For years I found that every time my notebook todo list ran out of space and I rewrote it I was much more productive that day – plus it was a cathartic experience. Arguably rewriting my list everyday is a waste of time because some items carry over and some items repeat. But the actual process of doing it, the planning rather than the plan, creates focus and motivation.

As you might have guess by now, a lot of this carrie directly over to my clients and their teams: prioritise the work, limit wip, let work fall off, stick to the system and the difficulties of discipline. Of course, what I’m describing is a system that emphasises planning over plans.

Another issue I regularly run up against is the “second priority” problem. Once all the pressing, really important and urgent stuff is done where do I put my attention? When I have three or four lower priority things to do it can be hard to choose which one to do and to stick with it. It can help to time-box the work, write “45” next to the work item and when I start work on it set a timer to stop after 45 minutes. I may not have done everything but I will have made progress and will at least have broken the second priority logjam.

Sometimes it’s hard to address an issue because it not clear what needs doing. When todo items are transactional “call X”, “write Y” it is easy to close them off. But sometimes hard to know what “Website” actually involves, I’m the one who decided I should work on my website, I’m the one doing the work, but what exactly should I be doing? And even if I know I should be, say: “updating keywords” what keywords? where? Even thought I can see something needs attention I don’t know what.

So, am I organized? or disorganized?

Well, I think this goes back to my dyslexia. Dyslexics are frequently disorganised because many (including me) suffer with poor short term memory. Left to myself I can be very disorganized.

But, dyslexics over compensate. A reoccurring pattern with dyslexics is that we have to create our own learning strategies and solutions to our own problems. Sometimes we over compensate and something we are bad at naturally becomes something we are very good at.

That I think is why other people think I’m really well organized and I think I’m badly organized.

And it is those ways of thinking and approaching organization – and work to do – which carries over to my professional work. Call it neurodiversity if you like.

Subscribe to my blog newsletter and download Continuous Digital for free

The post How I prioritise planning over plans appeared first on Allan Kelly, Software Strategy.

Online User Stories tutorials now complete

Allan Kelly from Allan Kelly, Software Strategy

Better User Stories
As a Product Owner I want to write better stories

I’m pleased to announce I’ve released the last of my online User Stories tutorials (part 5: Workflow and Lifecycle) and with that the whole series is complete. You can now buy the entire User Stories set of 5 tutorials as one package at a 40% discount to buying the tutorials individually.

The package includes over six hours of video commentary, exercises, quizzes, downloads and both ebook and audio book versions of Little Book of Requirements and User Stories.

Blog readers can get a further 25% off the price with the code: “blogoct21” until the end of this month, October 2021.

In addition, the first 3 people to use that code will receive a free print copy of The Art of Agile Product Ownership.

The 5 tutorials are:

These tutorials turned ouit to be a lot more work than I expected (where have i heard that before?). The core material is based on the Requirements, Backlogs and User Stories workshop that I have been running for a few years and last year converted to a series of online webinars. In the process the material has become a lot more focused.

Please, let me know what you think, in the comments section below or in the feedback forms at the end of each tutorial in the series.

The post Online User Stories tutorials now complete appeared first on Allan Kelly, Software Strategy.

Looking for a measurable impact from developer social learning

Derek Jones from The Shape of Code

Almost everything you know was discovered/invented by other people. Social learning (i.e., learning from others) is the process of acquiring skills by observing others (teaching is explicit formalised sharing of skills). Social learning provides a mechanism for skills to spread through a population. An alternative to social learning is learning by personal trial and error.

When working within an ecosystem that changes slowly, it is more cost-effective to learn from others than learn through trial and error (assuming that experienced people are available to learn from, and the learner is capable of identifying them); “Social Learning” by Hoppitt and Layland analyzes the costs and benefits of using social learning.

Since its inception, much of software engineering has been constantly changing. In a rapidly changing ecosystem, the experience of established members may suggest possible solutions that do not deliver the expected results in a changed world, i.e., social learning may not be a cost-effective way of building a skill set applicable within the new ecosystem.

Opportunities for social learning occur wherever developers tend to congregate.

When I started writing software people, developers would print out a copy of their code to take away and correct/improve/add-to (this was when 100+ people were time-sharing on a computer with 256K words of memory, running at 1 MHz). People would cluster around the printer, which ran sufficiently slowly that it was possible, in real-time, to read the code and figure out what was going on; it was possible to learn from others code (pointing out mistakes in programs that people planned to hand in was not appreciated). Then personal computers became available, along with low-cost printers (e.g., dot matrix), which were often shared, and did not print so fast that an experienced developer could not figure things out in real-time. Then laser printers came along, delivering a page at a time every 15 seconds, or so; experiencing the first print out from a Laser printer, I immediately knew that real-time code reading was a thing of the past (also, around this time, full-screen editors achieved the responsiveness needed to enthral developers, paper code listings could not compete). A regular opportunity for social learning disappeared.

Mentoring and retrospectives are intended as explicit (perhaps semi-taught) learning contexts, in which social learning opportunities may be available.

The effectiveness of social learning is dependent on being able to select a good enough source of expertise to learn from. Choosing the person with the highest prestige is a common social selection technique; selecting web pages appearing on the first page of a Google search is actually a form of conformist learning (i.e., selecting what others have chosen).

It is possible to point at particular instances of social learning in software engineering, but to what extent does social learning, other than explicit teaching, contribute to developer skills?

Answering this question requires enumerating all the non-explicitly taught skills a developer uses to get the job done, excluding the non-developer specific skills. A daunting task.

Is it even possible to consistently distinguish between social learning (implicit or taught) and individual learning?

For instance, take source code indentation. Any initial social learning is likely to have been subsequently strongly influenced by peer pressure, and default IDE settings.

Pronunciation of operator names is a personal choice that may only ever exist within a developer’s head. In my head, I pronounce the ^ operator as up-arrow, because I first encountered its use in the book Algorithms + Data Structures = Programs which used the symbol , which appears as the ^ character on modern keyboards. I often hear others using the word caret, which I have to mentally switch over to using. People who teach themselves to program have to invent names for unfamiliar symbols, until they hear somebody speaking code (the widespread availability of teach-yourself videos will make it rare to need for this kind of individual learning; individual learning is giving way to social learning).

The problem with attempting to model social learning is that much of the activity occurs in private, and is not recorded.

One public source of prestigious experience is Stack Overflow. Code snippets included as part of an answer on Stack Overflow appear in around 1.8% of Github repositories. However, is the use of this code social learning or conformist transmission (i.e., copy and paste)?

Explaining social learning to people is all well and good, but having to hand wave when asked for a data-driven example is not good. Suggestions welcome.

On Triumvirate – student

student from thus spake a.k.

When last they met, the Baron invited Sir R----- to join him in a wager involving a sequence of coin tosses. At a cost of seven coins Sir R----- would receive one coin for every toss of the coin until a run of three heads or of three tails brought the game to its conclusion.

To evaluate its worth to Sir R----- we begin with his expected winnings after a single toss of the coin.

Letter to my MP about climate emergency

Andy Balaam from Andy Balaam's Blog

[Introduction including details about my own air source heat pump install, and mention of the ending of the RHI funding in April 2022.]

After I have installed an air source heat pump, I will pay more money to heat my home, even though I am using less energy, because electricity is more expensive than gas. So this change will hurt me financially over both the short and longer terms.

Do you agree with me that climate emergency is the most important issue the government is now facing?

Do you also agree with me that we urgently need people to switch their heating and home insulation to reduce our dependence on burning gas?

Please do all you can to persuade the Prime Minister to introduce initiatives before COP26 that make it financially viable for families without spare cash to insulate their home and heat them with renewable energy.

Please pass my letter on to the Prime Minister and any government departments you consider relevant.

Thank you very much for your time.

The difference between no move constructor and a deleted move constructor

Anders Schau Knatten from C++ on a Friday

It’s easy to think that deleting the move constructor means removing it. So if you do MyClass(MyClass&&) = delete , you make sure it doesn’t get a move constructor. This is however not technically correct. It might seem like a nitpick, but it actually gives you a less useful mental model of what’s going on.

First: When does this matter? It matters for understanding in which cases you’re allowed to make a copy/move from an rvalue.

Here are some examples of having to copy/move an object of type MyClass:

MyClass obj2(obj1);
MyClass obj3(std::move(obj1));

MyClass obj4 = obj1;
MyClass obj5 = std::move(obj1);

return obj1;
return std::move(obj1);

These are all examples “direct initialization” (the former two) and “copy initialization” (the latter four). Note that there is no concept of “move initialization” in C++. Whether you end up using the copy or the move constructor to initialize the new object is just a detail. For the rest of this post, let’s just look at copy initialization, direct initialization works the same for our purposes. In any case you create a new copy of the object, and the implementation uses either the copy or the move constructor to do so.

Let’s first look at a class NoMove:

struct NoMove
    NoMove(const NoMove&);

This class has a user-declared copy constructor, so it doesn’t automatically get a move constructor. From the C++ standard https://timsong-cpp.github.io/cppwp/n4659/class.copy:

If the definition of a class X does not explicitly declare a move constructor, a non-explicit one will be implicitly declared as defaulted if and only if

– X does not have a user-declared copy constructor,
– (…)

So this class doesn’t have a move constructor at all. You didn’t explicitly declare one, and none got implicitly declared for you.

On the other hand, let’s see what happens if we explicitly delete the move constructor:

struct DeletedMove
    DeletedMove(const DeletedMove&);
    DeletedMove(DeletedMove&&) = delete;

This is called ” a deleted definition”. From the C++ standard: https://timsong-cpp.github.io/cppwp/n4659/dcl.fct.def.delete

A function definition of the form:
(…) = delete ;
is called a deleted definition. A function with a deleted definition is also called a deleted function.

Importantly, that does not mean that its definition has been deleted/removed and is no longer there. It means that is has a definition, and that this particular kind of definition is called a “deleted definition”. I like to read it as “deleted-definition”.

So our NoMove class has no move constructor at all. Our DeletedMove class has a move constructor with a deleted definition.

Why does this matter?

Let’s first look at a class with both a copy and a move constructor, and how to copy-initialize it.

struct Movable
    Movable(const Movable&);

Movable movable;
Movable movable2 = movable;

When initializing movable2, we need to find a function to do that with. A copy constructor would do nicely. And since we do have a copy constructor, it indeed gets used for this.

What if we turn movable into an rvalue?

Movable movable2 = std::move(movable);

Now a move constructor would be great. And we do have one, and it indeed gets used.

But what if we didn’t have a move constructor? That’s the case with our class NoMove from above.

struct NoMove
    NoMove(const NoMove&);

This one has a copy constructor, so it doesn’t get a move constructor. We can of course still make copies using the copy constructor:

NoMove noMove;
NoMove noMove2 = noMove;

But what happens now?

NoMove noMove;
NoMove noMove2 = std::move(noMove);

Are we now “move initializing” noMove2 and need the move constructor? Actually, we’re not. We’re still copy-initializing it, and need some function to do that task for us. A move constructor would be great, but a copy constructor would also do. It’s less efficient, but of course you’re allowed to make a copy of an rvalue.

So this is fine, the code compiles, and the copy constructor is used to make a copy of the rvalue.

What happened behind the scenes in all the examples above, is overload resolution. Overload resolution looks at all the candidates to do the job, and picks the best one. In the cases where we initialize from an lvalue, the only candidate is the copy constructor. We’re not allowed to move from an lvalue. In the cases where we initialize from an rvalue, both the copy and the move constructors are candidates. But the move constructor is a better match, as we don’t have to convert the rvalue to an lvalue reference. For Movable, the move constructor got selected. For NoMove, there is no move constructor, so the only candidate is the copy constructor, which gets selected.

Now, let’s look at what’s different when instead of having no move constructor, we have a move constructor with a deleted definition:

struct DeletedMove
    DeletedMove(const DeletedMove&);
    DeletedMove(DeletedMove&&) = delete;

We can of course still copy this one as well:

DeletedMove deletedMove = deletedMove;

But what happens if we try to copy-initialize from an rvalue?

DeletedMove deletedMove = std::move(deletedMove);

Remember, overload resolution tries to find all candidates to do the copy-initialization. And this class does in fact have both a copy and a move constructor, which are both candidates. The move constructor is picked as the best match, since again we avoid the conversion from an rvalue to an lvalue reference. But the move constructor has a deleted definition, and the program does not compile. From the C++ standard: https://timsong-cpp.github.io/cppwp/n4659/dcl.fct.def.delete#2:

A program that refers to a deleted function implicitly or explicitly, other than to declare it, is ill-formed. [ Note: This includes calling the function implicitly or explicitly (…) If a function is overloaded, it is referenced only if the function is selected by overload resolution.

The function is being called implicitly here, we’re not manually calling the move constructor. And we can see that this applies because overload resolution selected to use the move constructor with the deleted definition.

So the differences between not declaring a move constructor and defining one as deleted are:

  • The first one does not have a move constructor, the second one has a move constructor with a deleted definition.
  • The first one can be copy-initialized from an rvalue, the second can not.

If you enjoyed this post, you can subscribe to my blog, or follow me on Twitter.

Twenty Year Exit from the Oracle Ecosystem

Tim Pizey from Tim Pizey

The Oracle Ecosystem instance that I inheritted was designed from 2000 and went live in 2004. Its centre piece Oracle Workflow went end of life that year but had new versions through to 2007. In its own way it was a pinnacle of a certain view of software, the software vendor as a one stop shop, in the same way as DEC and IBM had previously sold their systems.
A whole set of components which were guaranteed to work together, with technical support. I probably would have made the same choice, even as late as 2000. This system was the largest system (by disk storage used) in Europe for a while. Housed in a purpose built computer room. Then came AWS S3 cloud storage.
The 'replacement' system migrated eighty percent of the files out of the database, seemingly unaware of the eighty-twenty rule, and duplicated the data and the dataflows. The replacement system was a 'microservice architecture' joined by a single datastore. The users now had two systems to use and were stuck with a 2000 vintage user interface. The next step was to stop using the Oracle Forms and introduce a modern three tier web application, choosing the new, shiny AngularJS as the front end, Hibernate on java as the middleware.
This was understandabley a big job, the UI was now on AWS but the data was in a data centre (a commercial one by now). The need to quit the data centre motivated the removal of CMSDK (Content Management Software Development Kit) from inside the database to separate, external, email and SFTP handling systems, to reduce databse size and enable security in the cloud.
The remaining steps are to finsh the migration of files (who knew this would be the difficult bit) and to migrate from Oracle AQ (Advanced Queues (software naming error 101: avoid hubristic adjectives)) to AWS SQS (Simple Queueing Service (software naming error 102: avoid indexical adjectives)). Note that we have to upgrade AngularJS to Angular just to stay still, and pop things into Kubernetes, just because.
This is clean, recogisable, manageable and stable. We could rest here for a while; but all that pain motivates completion:
The elephant is in the room. The final step is then to replace Oracle Workflow with Camunda.

Two failed software development projects in the High Court

Derek Jones from The Shape of Code

When submitting a bid, to be awarded the contract to develop a software system, companies have to provide information on costs and delivery dates. If the costs are significantly underestimated, and/or the delivery dates woefully optimistic, one or more of the companies involved may resort to legal action.

Searching the British and Irish Legal Information Institute‘s Technology and Construction Court Decisions throws up two interesting cases (when searching on “source code”; I have not been able to figure out the patterns in the results that were not returned by their search engine {when I expected some cases to be returned}).

The estimation and implementation activities described in the judgements for these two cases could apply to many software projects, both successful and unsuccessful. Claiming that the system will be ready by the go-live date specified by the customer is an essential component of winning a bid, the huge uncertainties in the likely effort required comes as standard in the software industry environment, and discovering lots of unforeseen work after signing the contract (because the minimum was spent on the bid estimate) is not software specific.

The first case is huge (BSkyB/Sky won the case and EDS had to pay £200+ million): (1) BSkyB Limited (2) Sky Subscribers Services Limited: Claimants – and (1) HP Enterprise Services UK Limited (formerly Electronic Data Systems Limited) (2) Electronic Data systems LLC (Formerly Electronic Data Systems Corporation: Defendants. The amount bid was a lot less than £200 million (paragraph 729 “The total EDS “Sell Price” was £54,195,013 which represented an overall margin of 27% over the EDS Price of £39.4 million.” see paragraph 90 for a breakdown).

What can be learned from the judgement for this case (the letter of Intent was subsequently signed on 9 August 2000, and the High Court decision was handed down on 26 January 2010)?

  • If you have not been involved in putting together a bid for a large project, paragraphs 58-92 provides a good description of the kinds of activities involved. Paragraphs 697-755 discuss costing details, and paragraphs 773-804 manpower and timing details,
  • if you have never seen a software development contract, paragraphs 93-105 illustrate some of the ways in which delivery/payments milestones are broken down and connected. Paragraph 803 will sound familiar to developers who have worked on large projects: “… I conclude that much of Joe Galloway’s evidence in relation to planning at the bid stage was false and was created to cover up the inadequacies of this aspect of the bidding process in which he took the central role.” The difference here is that the money involved was large enough to make it worthwhile investing in a court case, and Sky obviously believed that they could only be blamed for minor implementation problems,
  • don’t have the manager in charge of the project give perjured evidence (paragraph 195 “… Joe Galloway’s credibility was completely destroyed by his perjured evidence over a prolonged period.”). Bringing the law of deceit and negligent misrepresentation into a case can substantially increase/decrease the size of the final bill,
  • successfully completing an implementation plan requires people with the necessary skills to do the work, and good people are a scarce resource. Projects fail if they cannot attract and keep the right people; see paragraphs 1262-1267.

A consequence of the judge’s finding of misrepresentation by EDS is a requirement to consider the financial consequences. One item of particular interest is the need to calculate the likely effort and time needed by alternative suppliers to implement the CRM System.

The only way to estimate, with any degree of confidence, the likely cost of implementing the required CRM system is to use a conventional estimation process, i.e., a group of people with the relevant domain knowledge work together for some months to figure out an implementation plan, and then cost it. This approach costs a lot of money, and ties up scarce expertise for long periods of time; is there a cheaper method?

Management at the claimant/defence companies will have appreciated that the original cost estimate is likely to be as good as any, apart from being tainted by the perjury of the lead manager. So they all signed up to using Tasseography, e.g., they get their respective experts to estimate the amount of code that needs to be produce to implement the system, calculate how long it would take to write this code and multiply by the hourly rate for a developer. I would loved to have been a fly on the wall when the respective IT experts, all experienced in provided expert testimony, were briefed. Surely the experts all knew that the ballpark figure was that of the original EDS estimate, and that their job was to come up with a lower/high figure?

What other interpretation could there be for such a bone headed approach to cost estimation?

The EDS expert based his calculation on the debunked COCOMO model (ok, my debunking occurred over six years later, but others have done it much earlier).

The Sky expert based his calculation on the use of function points, i.e., estimation function points rather than lines of code, and then multiply by average cost per function point.

The legal teams point out the flaws in the opposing team’s approach, and the judge does a good job of understanding the issues and reaching a compromise.

There may be interesting points tucked away in the many paragraphs covering various legal issues. I barely skimmed these.

The second case is not as large (the judgement contains a third the number of paragraphs, and the judgement handed down on 19 February 2021 required IBM to pay £13+ million): SCIS GENERAL INSURANCE LIMITED: Claimant – and – IBM UNITED KINGDOM LIMITED: Defendant.

Again there is lots to learn about how projects are planned, estimated and payments/deliveries structured. There are staffing issues; paragraph 104 highlights how the client’s subject matter experts are stuck in their ways, e.g., configuring the new system for how things used to work and not attending workshops to learn about the new way of doing things.

Every IT case needs claimant/defendant experts and their collection of magic spells. The IBM expert calculated that the software contained technical debt to the tune of 4,000 man hours of work (paragraph 154).

If you find any other legal software development cases with the text of the judgement publicly available, please let me know (two other interesting cases with decisions on the British and Irish Legal Information Institute).