## User Stories by Example part 3 (Refactoring)

The latest instalment of my online User Stories tutorial is now available online, User Stories by Example: Refactoring.

It takes as its starting point some existing stories and reworks them to convey their message more clearly. In the process I discuss:

The use of time-boxed spikes.

The naming of team members in user stories, e.g. “As a developer I want …” – and why this isn’t a good idea.

Rewriting user stories and breaking them down into more smaller stories. (More on this in the next tutorial.)

Why more smaller stories is better than a fewer larger stories.

How acceptance criteria can be used to split stories into smaller pieces.

and a brief look at dealing with dependencies.

Videos are intersperced with exercises and quizzes. My guess is this tutorial will take two to three hours to complete – which can be all in one go or split over days or weeks to suit yourself. As with the earlier tutorials I work through real life user stories to illustrate and draw lessons.

This is the third tutorial, it joins User Stories by Example part 1 Starting with Stories and part 2 Acceptance Criteria. The next module will look at splitting stories in more detail.

The tutorial this carries the introductory price of \$49. In time this price will probably rise and I’ll introduce a combined option to buy all the courses in one go.

The post User Stories by Example part 3 (Refactoring) appeared first on Allan Kelly Associates.

## Estimating using a granular sequence of values

When asked for an estimate of the time needed to complete a task, should developers be free to choose any numeric value, or should they be restricted to selecting from a predefined set of values (e.g, the Fibonacci numbers, or T-shirt sizes)?

Allowing any value to be chosen would appear to provide the greatest flexibility to make an accurate estimate. However, estimating is an intrinsically uncertain process (i.e., the future is unknown), and it is done by people with varying degrees of experience (which might be used to help guide their prediction about the future).

Restricting the selection process to one of the values in a granular sequence of numbers has several benefits, including:

• being able to adjust the gaps between permitted values to match the likely level of uncertainty in the task effort, or the best accuracy resolution believed possible,
• reducing the psychological stress of making an estimate, by explicitly giving permission to ignore the smaller issues (because they are believed to require a total effort that is less than the sequence granularity),
• helping to maintain developer self-esteem, by providing a justification when an estimate turning out to be inaccurate, e.g., the granularity prevented a more accurate estimate being made.

Is there an optimal sequence of granular values to use when making task estimates for a project?

The answer to this question depends on what is attempting to be optimized.

Given how hard it is to get people to produce estimates, the first criterion for an optimal sequence has to be that people are willing to use it.

I have always been struck by the ritualistic way in which the Fibonacci sequence is described by those who use it to make estimates. Rituals are an effective technique used by groups to help maintain members’ adherence to group norms (one of which might be producing estimates).

A possible reason for the tendency to use round numbers might estimate-values is that this usage is common in other social interactions involving numeric values, e.g., when replying to a request for the time of day.

The use of round numbers, when developers have the option of selecting from a continuous range of values, is a developer imposed granular sequence. What form do these round number sequences take?

The plot below shows the values of each of the six most common round number estimates present in the BrightSquid, SiP, and CESAW (project 615) effort estimation data sets, plus the first six Fibonacci numbers (code+data):

The lines are fitted regression models having the form: (there is a small variation in the value of the constant; the smallest value for project 615 was probably calculated rather than being human selected).

This plot shows a consistent pattern of use across multiple projects (I know of several projects that use Fibonacci numbers, but don’t have any publicly available data). Nothing is said about this pattern being (near) optimal in any sense.

The time unit of estimation for this data was minutes or hours. Would the equation have the same form if the time unit was days, would the constant still be around . I await the data needed to answer this question.

This brief analysis looked at granular sequences from the perspective of the distribution of estimates made. Perhaps it makes more sense to base a granular estimation sequence on the distribution of actual task effort. A topic for another post.

## Estimating using a granular sequence of values

When asked for an estimate of the time needed to complete a task, should developers be free to choose any numeric value, or should they be restricted to selecting from a predefined set of values (e.g, the Fibonacci numbers, or T-shirt sizes)?

Allowing any value to be chosen would appear to provide the greatest flexibility to make an accurate estimate. However, estimating is an intrinsically uncertain process (i.e., the future is unknown), and it is done by people with varying degrees of experience (which might be used to help guide their prediction about the future).

Restricting the selection process to one of the values in a granular sequence of numbers has several benefits, including:

• being able to adjust the gaps between permitted values to match the likely level of uncertainty in the task effort, or the best accuracy resolution believed possible,
• reducing the psychological stress of making an estimate, by explicitly giving permission to ignore the smaller issues (because they are believed to require a total effort that is less than the sequence granularity),
• helping to maintain developer self-esteem, by providing a justification when an estimate turning out to be inaccurate, e.g., the granularity prevented a more accurate estimate being made.

Is there an optimal sequence of granular values to use when making task estimates for a project?

The answer to this question depends on what is attempting to be optimized.

Given how hard it is to get people to produce estimates, the first criterion for an optimal sequence has to be that people are willing to use it.

I have always been struck by the ritualistic way in which the Fibonacci sequence is described by those who use it to make estimates. Rituals are an effective technique used by groups to help maintain members’ adherence to group norms (one of which might be producing estimates).

A possible reason for the tendency to use round numbers might estimate-values is that this usage is common in other social interactions involving numeric values, e.g., when replying to a request for the time of day.

The use of round numbers, when developers have the option of selecting from a continuous range of values, is a developer imposed granular sequence. What form do these round number sequences take?

The plot below shows the values of each of the six most common round number estimates present in the BrightSquid, SiP, and CESAW (project 615) effort estimation data sets, plus the first six Fibonacci numbers (code+data):

The lines are fitted regression models having the form: (there is a small variation in the value of the constant; the smallest value for project 615 was probably calculated rather than being human selected).

This plot shows a consistent pattern of use across multiple projects (I know of several projects that use Fibonacci numbers, but don’t have any publicly available data). Nothing is said about this pattern being (near) optimal in any sense.

The time unit of estimation for this data was minutes or hours. Would the equation have the same form if the time unit was days, would the constant still be around . I await the data needed to answer this question.

This brief analysis looked at granular sequences from the perspective of the distribution of estimates made. Perhaps it makes more sense to base a granular estimation sequence on the distribution of actual task effort. A topic for another post.

## On The Fours Reawakens – student

Recall that the Baron's latest game involved he and Sir R----- each casting four four sided dice. For each pair of dice formed from one of the Baron's and one of Sir R-----'s with matching faces, Sir R----- should have received thirteen coins having given twenty eight to play.

## What is known about software effort estimation in 2021

What do we know about software effort estimation, based on evidence?

The few publicly available datasets (e.g., SiP, CESAW, and Renzo) involve (mostly) individuals estimating short duration tasks (i.e., rarely more than a few hours). There are other tiny datasets, which are mostly used to do fake research. The patterns found across these datasets include:

• developers often use round-numbers,
• the equation: , where is a constant that varies between projects, often explains around 50% of the variance present in the data. This equation shows that developers under-estimate short tasks and over-estimate long tasks. The exponent, , applies across most projects in the data,
• individuals tend to either consistently over or under estimate,
• developer estimation accuracy does not change with practice. Possible reasons for this include: variability in the world prevents more accurate estimates, developers choose to spend their learning resources on other topics.

Does social loafing have an impact on actual effort? The data needed to answer this question is currently not available (the available data mostly involves people working on their own).

When working on a task, do developers follow Parkinson’s law or do they strive to meet targets?

The following plot suggests that one or the other, or both are true (data):

On the left: Each colored lines shows the number of tasks having a given actual implementation time, when they were estimated to take 30, 60 or 120 minutes (the right plot reverses the role of estimate/actual). Many of the spikes in the task counts are at round numbers, suggesting that the developer has fixated on a time to finish and is either taking it easy or striving to hit it. The problem is distinguishing them mathematically; suggestions welcome.

None of these patterns of behavior appear to be software specific. They all look like generic human behaviors. I have started emailing researchers working on project analytics in other domains, asking for data (no luck so far).

Other patterns may be present for many projects in the existing data, we have to wait for somebody to ask the right question (if one exists).

It is also possible that the existing data has some unusual characteristics that don’t apply to most projects. We won’t know until data on many more projects becomes available.

## Turning this particular server into a bit less of a pet

I’m in the middle of a server redo - right now, I’m setting up a replacement server for my trusty Dell T30, plus it was time to give this web server a new home. When I started the migration from my old WordPress site to the new static site, the static site was running on a small 1 core / 1GB RAM cloud server at Vultr. That had enough oomph for testing and for the last couple of months.

## Software is not a lump or work to do (and the laws which prove it)

Building software is not like digging a hole in the ground: your target changes as you dig, the people and machines you use change the outcome as much as any original idea, and you never really know when to stop digging.

Years ago I was hired by Reuters to build an interface connector between the Liffe trading exchange and their data systems. Another developer started a few weeks after me. In the end we produced about half a dozen modules that worked together to make the connection. But we only produced that many because we were over staffed. In reality the one module I wrote handled 90% of the work required, the other modules were largely superfluous. And certainly the extra time which would have been required to make my module do 100% of the work was less than the time I spent to make sure it worked with the other.

Regular readers have probably already recognised Kelly’s Second Law of software: Inside every large development effort there is a small one struggling to get out

This itself is an example of Parkinson’s Law: work expands so as to fill the time available for its completion.

Actually you might restate my second law as Parkinson’s Law in reverse: constraining the capacity to do work reduces the amount of work to do. If you think about it this is a result of Conway’s Law: system designs copy the communication channels in the organization which creates the system.

Economists see a related phenomenon as the Lump of Work fallacy: people assume there is a fixed lump of work to do. If there are more people to do the work (say from immigration) then there will be unemployment – or possibly wages will be forced down and same work is distributed between more people. However it doesn’t national economies work like that, more people demand more food, houses, healthcare, schools and so on. What is true in the small is not true in the large. The net effect can be positive for countries but exactly how and by how much is hotly debated.

Software development has its own lump of work fallacy. There isn’t a fixed amount of work to do. Rather than saying “How many people will it take and how long will it take?” It is better to say “If we have five people working on this for three months what progress can we make?”

Writing software is not like digging a hole in the ground: the work to do is neither really known in advance nor is it fixed. Adding people actually increases the amount of work to do – Brooks’s Law.

At the start nobody really knows what it required, they may get lucky but more often that not once the thing is put in front of clients and (potential) customers the ask changes. I’ve heard this called Humphrey’s Law (after Watts Humphrey) although that name is not in common usage and there is another Humphrey’s Law from the world of psychology – which is connected with time estimation discussions for different reasons.

When you put Parkinson’s Law together with this “don’t know until I see it” law you get Hofstadter’s law: It always takes longer than you expect, even when you take into account Hofstadter’s Law.

Assume for the moment that one can know what is wanted in advance there is another problem: there are multiple ways to achieve the same result. There is no one true way in software development, there are always multiple ways to achieve the same end result.

Even if one was to fix the big questions – the OS, the delivery platform, the programming language and the database and so on – then you can still create very different implementations for the same thing. Or as I usually put it “there are many ways to dice the onion” – my Crown Jewels post describes how I can get wildly different time and money estimates for what is basically the same piece of work.

Of course the big consequence of these is estimation – how can anyone estimate in these circumstances? Let me go further and suggest that the process of estimating the work to do is more likely than not to increase the amount of work to do. Not only does estimation take time to do itself, but there when estimating there is a tendency to “play safe” and favour larger estimates even when these estimates are themselves underestimates (see Vierordt’s law.)

Sometimes it feels like quantum physics: when one parameter is measured another changes, we can know the speed but not the direction, or the direction but not the speed.

I’m not sure I have a complete answer but I have some of the pieces.

Start new work with a Minimally Viable Team: task the team to start immediately and in parallel work to understand what is needed and create potential solutions.

Keep teams stable: This contains a lot of variables and provides some past performance data to include in calculations. It will at times be necessary for the teams to call in more help – pull more skills and resources as needed.

By working with existing and minimally viable teams the problems are partially constrained: the technologies available are mostly the technologies the team knows already, the number of people available to work on the work is the number of people on the team. In time you may change both these parameters but initially they are constraints to work within.

Use active portfolio management and governance to kill work which is under performing or escalating beyond expectations.

The next time someone says “Building software should be like building a house” please remind them you aren’t building a house. What software engineers do is massively more variable and complex than building another example of the same thing.

Back at Reuters, the bright side was that the over engineered system we built wasn’t just used for Liffe, it was used to connect 2 or 3 other exchanges too so maybe over staffing was worth it. Except I don’t believe it really made economic sense, it would have been better to get the one exchange working and only add the minimum to the system when the second and third exchanges came along – diseconomies of scale again.

The post Software is not a lump or work to do (and the laws which prove it) appeared first on Allan Kelly Associates.

## Visual Studio 2022 Preview 1 is now available

Ever since news broke in April of Microsoft's plans for Visual Studio 2022 we have been waiting to get our hands on a preview build of the new version.

The first public preview was finally released on 17th June, so naturally we have spent the past few days taking a look to see what we need to do - both to analyse codebases using it and to update our existing Visual Lint and ResOrg plugins to integrate within it.

From what I can see, Visual Studio 2022 Preview 1 seems to use Visual Studio 2019 project files (the platform toolset is still `v142`, and `_MSC_VER` is still 1929). I've no doubt that will change in subsequent preview versions.

However, unlike its predecessor Visual Studio 2019, Visual Studio 2022 is a fully 64 bit development environment (although VS2019 and its predecessors can compile and debug 64 bit projects, the IDEs themselves are firmly 32 bit).

As such it follows that plugins for Visual Studio 2022 must also be compiled as 64 bit. Although the plug-in architecture is not changing in VS2022 (see The Future of Visual Studio Extensions), the move to 64 bit is a very significant change in itself.

For our Visual Studio plugin to run within Visual Studio 2022, what we have to do is basically:

• Recompile the existing plugins to target the x64 platform, using the new (VS2022 specific) 64 bit Visual Studio COM interfaces and the new (and also VS2022 specific) version of the Visual Studio SDK.

• Add Visual Studio 2022 specific VSIX packages to deploy the x64 version of the plugins.

• Update the installers to recognise Visual Studio 2022 and install the new x64 version of the plugin.

• Try really, really hard not to break support for earlier Visual Studio versions in the process (those cannot load 64 bit plugins, so the existing 32 bit versions still have to be maintained).

The full details of all of the changes involved are somewhat beyond the scope of this blogpost, but suffice it to say we have already started preparing to port the code to Visual Studio 2022 and expect that to keep us busy over the summer.

(well, it's not as if anyone can really go on holiday right now, is it?)

The screenshots below show Visual Studio 2022 Preview 1 with an AppWizard generated C++ VSIX extension project loaded and running. Note that I had to make a couple of minor corrections to the generated project and property files to get it to compile and link.

An AppWizard generated VS2022 extension project loaded inside Visual Studio 2022 Preview 1. An AppWizard generated VS2022 extension project running inside an experimental instance of Visual Studio 2022 Preview 1.

## Visual Studio 2022 Preview 1

Ever since news broke in April of Microsoft's plans for Visual Studio 2022 we have been waiting to get our hands on a preview build of the new version.

The first public preview was finally released on 17th June, so naturally we have spent the past few days taking a look to see what we need to do - both to analyse codebases using it and to update our existing Visual Lint and ResOrg plugins to integrate within it.

From what I can see, Visual Studio 2022 Preview 1 seems to use Visual Studio 2019 project files (the platform toolset is still v142, and `_MSC_VER` is still 1929). I've no doubt that will change in subsequent preview versions.

However, unlike its predecessor Visual Studio 2019, Visual Studio 2022 is a fully 64 bit development environment (although VS2019 and its predecessors can compile and debug 64 bit projects, the IDEs themselves are firmly 32 bit).

As such it follows that plugins for Visual Studio 2022 must also be compiled as 64 bit. Although the plug-in architecture is not changing in VS2022 (see The Future of Visual Studio Extensions), the move to 64 bit is a very significant change in itself.

For our Visual Studio plugin to run within VS2022, what we have to do is basically:

• Recompile the existing plugins to target the x64 platform, using the new (VS2022 specific) 64 bit Visual Studio COM interfaces and the new (and also VS2022 specific) version of the Visual Studio SDK.

• Add Visual Studio 2022 specific VSIX packages to deploy the x64 version of the plugins.

• Update the installers to recognise VS2022 and install the new x64 version of the plugin.

• Try not to break support for earlier Visual Studio versions in the process (those cannot load 64 bit plugins, so the existing 32 bit versions will still have to be maintained).

The full details of all of the changes involved are somewhat beyond the scope of this blogpost, but suffice it to say we have already started preparing to port the code to VS2022 and expect that to keep us busy over the summer.

(well, it's not as if anyone can really go on holiday right now, is it?)

The screenshots below show VS2022 Preview 1 with an AppWizard generated C++ VSIX extension project loaded and running. Note that I had to make a couple of minor corrections to the generated project and property files to get it to compile and link.

An AppWizard generated VS2022 extension project loaded inside Visual Studio 2022 Preview 1. An AppWizard generated VS2022 extension project running inside an experimental instance of Visual Studio 2022 Preview 1.