Unneeded requirements implemented in Waterfall & Agile

Derek Jones from The Shape of Code

Software does not wear out, but the world in which it runs evolves. Time and money is lost when, after implementing a feature in software, customer feedback is that the feature is not needed.

How do Waterfall and Agile implementation processes compare in the number of unneeded feature/requirements that they implement?

In a Waterfall process, a list of requirements is created and then implemented. The identity of ‘dead’ requirements is not known until customers start using the software, which is not until it is released at the end of development.

In an Agile process, a list of requirements is used to create a Minimal Viable Product, which is released to customers. An iterative development processes, driven by customer feedback, implements requirements, and makes frequent releases to customers, which reduces the likelihood of implementing known to be ‘dead’ requirements. Previously implemented requirements may be discovered to have become ‘dead’.

An analysis of the number of ‘dead’ requirements implemented by the two approaches appears at the end of this post.

The plot below shows the number of ‘dead’ requirements implemented in a project lasting a given number of working days (blue/red) and the difference between them (green), assuming that one requirement is implemented per working day, with the discovery after 100 working days that a given fraction of implemented requirements are not needed, and the number of requirements in the MVP is assumed to be small (fractions 0.5, 0.1, and 0.05 shown; code):

Dead requirements for Waterfall and Agile projects running for a given number of days, along with difference between them.

The values calculated using one requirement implemented per day scales linearly with requirements implemented per day.

By implementing fewer ‘dead’ requirements, an Agile project will finish earlier (assuming it only implements all the needed requirements of a Waterfall approach, and some subset of the ‘dead’ requirements). However, unless a project is long-running, or has a high requirements’ ‘death’ rate, the difference may not be compelling.

I’m not aware of any data on rate of discovery of ‘dead’ implemented requirements (there is some on rate of discovery of new requirements); as always, pointers to data most welcome.

The Waterfall projects I am familiar with, plus those where data is available, include some amount of requirement discovery during implementation. This has the potential to reduce the number of ‘dead’ implemented requirements, but who knows by how much.

As the size of Minimal Viable Product increases to become a significant fraction of the final software system, the number of fraction of ‘dead’ requirements will approach that of the Waterfall approach.

There are other factors that favor either Waterfall or Agile, which are left to be discussed in future posts.

The following is an analysis of Waterfall/Agile requirements’ implementation.

Define:

F_{live} is the fraction of requirements per day that remain relevant to customers. This value is likely to be very close to one, e.g., 0.999.
R_{done} requirements implemented per working day.

Waterfall

The implementation of R_{total} requirements takes I_{days}=R_{total}/R_{done}days, and the number of implemented ‘dead’ requirements is (assuming that the no ‘dead’ requirements were present at the end of the requirements gathering phase):

R_{Wdead}=R_{total}*(1-{F_{live}}^{I_{days}})

As I_{days} right infty effectively all implemented requirements are ‘dead’.

Agile

The number of implemented ‘live’ requirements on day n is given by:

R_n=F_{live}*R_{n-1}+R_{done}

with the initial condition that the number of implemented requirements at the start of the first day of iterative development is the number of requirements implemented in the Minimum Viable Product, i.e., R_0=R_{mvp}.

Solving this difference equation gives the number of ‘live’ requirements on day n:

R_n=R_{mvp}*{F_{live}}^n+{n*R_{done}}/{n(1-F_{live})+F_{live}}

as n right infty, R_n approaches to its maximum value of {R_{done}}/{1-F_{live}}

Subtracting the number of ‘live’ requirements from the total number of requirements implemented gives:

R_{Adead}=R_{mvp}+n*R_{done}-R_n

or

R_{Adead}=R_{mvp}(1-{F_{live}}^n)+n*R_{done}(1-1/{n(1-F_{live})+F_{live}})
or
R_{Adead}=R_{mvp}(1-{F_{live}}^n)+n*R_{done}{n-1}/{n+F_{live}/(1-F_{live})}

as n right infty effectively all implemented requirements are ‘dead’, because the number of ‘live’ requirements cannot exceed a known maximum.

Optimal sizing of a product backlog

Derek Jones from The Shape of Code

Developers working on the implementation of a software system will have a list of work that needs to be done, a to-do list, known as the product backlog in Agile.

The Agile development process differs from the Waterfall process in that the list of work items is intentionally incomplete when coding starts (discovery of new work items is an integral part of the Agile process). In a Waterfall process, it is intended that all work items are known before coding starts (as work progresses, new items are invariably discovered).

Complaints are sometimes expressed about the size of a team’s backlog, measured in number of items waiting to be implemented. Are these complaints just grumblings about the amount of work outstanding, or is there an economic cost that increases with the size of the backlog?

If the number of items in the backlog is too low, developers may be left twiddling their expensive thumbs because they have run out of work items to implement.

A parallel is sometimes drawn between items waiting to be implemented in a product backlog and hardware items in a manufacturer’s store waiting to be checked-out for the production line. Hardware occupies space on a shelf, a cost in that the manufacturer has to pay for the building to hold it; another cost is the interest on the money spent to purchase the items sitting in the store.

For over 100 years, people have been analyzing the problem of the optimum number of stock items to order, and at what stock level to place an order. The economic order quantity gives the optimum number of items to reorder, Q (the derivation assumes that the average quantity in stock is Q/2), it is given by:

Q=sqrt{{2DK}/h}, where D is the quantity consumed per year, K is the fixed cost per order (e.g., cost of ordering, shipping and handling; not the actual cost of the goods), h is the annual holding cost per item.

What is the likely range of these values for software?

  • D is around 1,000 per year for a team of ten’ish people working on multiple (related) projects; based on one dataset,
  • K is the cost associated with the time taken to gather the requirements, i.e., the items to add to the backlog. If we assume that the time taken to gather an item is less than the time taken to implement it (the estimated time taken to implement varies from hours to days), then the average should be less than an hour or two,
  • h: While the cost of a post-it note on a board, or an entry in an online issue tracking system, is effectively zero, there is the time cost of deciding which backlog items should be implemented next, or added to the next Sprint.

    If the backlog starts with n items, and it takes t seconds to decide whether a given item should be implemented next, and f is the fraction of items scanned before one is selected: the average decision time per item is: avDecideTime={f*n*(f*n+1)/2}*t seconds. For example, if n=50, pulling some numbers out of the air, f=0.5, and t=10, then avDecideTime=325, or 5.4 minutes.

    The Scrum approach of selecting a subset of backlog items to completely implement in a Sprint has a much lower overhead than the one-at-a-time approach.

If we assume that K/h==1, then Q=sqrt{2*1000}=44.7.

An ‘order’ for 45 work items might make sense when dealing with clients who have formal processes in place and are not able to be as proactive as an Agile developer might like, e.g., meetings have to be scheduled in advance, with minutes circulated for agreement.

In a more informal environment, with close client contacts, work items are more likely to trickle in or appear in small batches. The SiP dataset came from such an environment. The plot below shows the number of tasks in the backlog of the SiP dataset, for each day (blue/green) and seven-day rolling average (red) (code+data):

Tasks waiting to be implemented, per day, over duration of SiP projects.

Books to be written

Allan Kelly from Allan Kelly

I’m entering the final stretch of writing for my new book so it is time to think hard about the name.

“How I write books” was only ever a working title, now I think I’ve hit on the permanent name “Books to be written“. What do you think?

Let me know – you can check out the (nearly finished) book on LeanPub right now. Almost all the chapters are now drafted.

The post Books to be written appeared first on Allan Kelly.

How fresh is your backlog?

Allan Kelly from Allan Kelly

Do you struggle with an overwhelming backlog?

Do you count the number of product backlog items in your backlog in tens? hundred? or thousands?

Does your backlog contain many stories which have been there for months, if not years, and yet never raise to the top of the backlog?

Is your success judged on your ability to do the backlog?

Backlogs were a good idea when they were introduced a bit over 20 years ago but today many teams slaves to the backlog – see my posts on the Tyranny of the backlog and Purpose over backlog. One of the benefits I’ve called out for OKRs is the ability to move away from backlog driven development (BLDD).

In Succeeding with OKRs in Agile I ask suggest you either need to prioritise your backlog over OKRs (in which case OKRs are derived from the backlog you intend to do) or OKRs over backlog (in which case OKRs are derived from strategy and the backlog plays a supporting role.) In my podcast with Jenny Herald earlier this year I even say “Let OKRs drive… nuke the backlog.”

Filipe Albero Pomar recently shared his backlog freshness blog which I think is great. Freshness is a great way to think about the state of the backlog that separates the size of the backlog from the relevance of the backlog.

Filipe’s idea is simply to talk about the backlog in terms of freshness – you have a fresh backlog if your backlog items are fresh: written recently, relate to current opportunities, problems and things people currently want.

And of course, the opposite of fresh: stale, stories that have been sitting in the backlog for months, even years, stories that relate to yesterday’s problems and project, stories which people wanted last year. The existence a big backlog of stale stories means the team is seen to be not delivering, the end-date is far off because people still expect all the work to be done.

Filipe suggests backlog freshness can be measured:

1. Set a cut-off date

2. Categorise stories as fresh or stale: fresh stories have been written since the cut-off date, those which are older are stale

3. Calculate freshness as a percentage of fresh from the total: if 25 stories out of 60 have been written in the last month then the backlog is 41% fresh, and 59% stale

Thats a useful metric, I think we can do better, look at the graphic above. I group backlog items into age groups and graphed them. For completeness I added a line to indicate average story age. Clearly this backlog is not fresh – nearly half the stories are over a year old.

I like the idea of graphing backlog freshness because it is easy to understand and makes an impact. In the graph above I’ve categorised backlog items into age groups and added an average line. Clearly this is not a fresh backlog. Whether this is the way to demonstrate backlog freshness I’m not sure – I’m playing with a histogram and quartile ranges.

With some clients I’ve thought of the backlog like a mortgage. There is the principle (the existing backlog), the interest rate (the growth rate of the backlog) and the monthly repayments (stories reaching done). Unfortunately when you do this you sometimes find the mortgage will not be paid for many years, and perhaps never. (Don’t worry about estimating the size of stories, for this sort of analysis the number of stories will get you started, and if your backlog is measured in hundreds of items the small will offset the large.)

I’d love to talk more about this and experiment with some ideas, I think it could be a very useful way of thinking about the backlog.


Subscribe to my blog newsletter and get Continuous Digital for free

The post How fresh is your backlog? appeared first on Allan Kelly.

Scrum or OKRs, which comes first?

Allan Kelly from Allan Kelly

“Hello Allan , I followed a few of your seminars on OKRs and I am reading your book.  I do have one question. I will start a transformation soon. The company has the ambition to move to Scrum and also OKRs; Do you have any advise on where to start? OKRs first or SCRUM first.  Thank you a lot !”

This question appeared in my mailbox – or rather my LinkedIn box – a few weeks ago and I thought other readers might be interested in my answer.

My answer is: do both together.

That will sound ambitious if you see Scrum and OKRs as distinct things but I don’t, to me they are synergistic and can be viewed as one change. In the process I see the opportunity to create a simpler form of agile, or simpler version of Scrum if you prefer. Let me describe.

The Scrum Master role remains pretty much the same as regular Scrum. The main change is that in addition to facilitating planning meetings and sprint retrospectives the Scrum Master will additionally facilitate OKR setting sessions and OKR retrospectives at the end of the cycle. They will want to ensure the OKRs are known by everyone and people reference to them during sprints.

Similarly the Product Owner role remains pretty much the same with the additional need for the PO to lead thinking on what OKRs to set. This means the Product Owner needs to do more horizon scanning, talk to more stakeholders and prepare to set objectives that will last several sprints rather than just one at a time.

Simplification comes in that there is no need to build up and administer a product backlog. Set the OKRs, then go straight into a planning meeting and ask “How do we make progress against this OKR?”. Write the stories you need to do that. Use the OKRs as a just-in-time story generator. If you generate stories you cannot do this sprint then put them in a product backlog for the next planning meeting.

In the next planning meeting review progress and ask yourself again: “what do we need to do to make progress on these OKRs?”. It might be that you have some stories from last time to work with, if not write some more. OKRs in effect become the Sprint Goal and you generate the stories from there. (If you are using a Product Goal as well then reference that when setting the OKRs at the start of the cycle.)

Estimation is reduced because you have fewer stories in play and in the backlog. If you want you can use story points and velocity to determine if the sprint is full or you can just ask the team “Is that enough work to keep us busy?” and “Do we have space for more?”

If you want a more sophisticated system then get the stories out on the table, put them in the order they are likely to be done, then go from top to bottom asking “On a scale of 1 to 10, What are the chances we will get this story done in this sprint? – where 1 is unlikely and 10 is almost certainly.” If the probability is below 8 you probably take action, break the story down, reorder the work order or just accept that you probably won’t do everything you would like to.

In the longer term you can count the stories done to build up a record of capacity.

I would aim to do end of sprint demonstrations and better still release to live. The OKR may not be complete but the stories in the sprint can start delivering value early. As usual keep quality high, aim for zero bugs and automate everything that needs testing.

If you are new to both Scrum and OKRs then I would probably run one week sprints and a six or eight-week OKR cycle the first time. Do a retrospective at the end of the OKR cycle, after that you might move to two-week sprints and 12 week OKR cycles but keep an open mind and decide what works for you.

All the way through keep delivering benefit to customers, hitting OKRs is icing on the cake, and there are no prizes for doing perfect Scrum.

I’d like to think I outlined this recipe in Succeeding with OKRs in Agile but I suspect I wasn’t as clear as I could have been. Increasingly I’m seeing OKRs as a means of stripping agile back and simplifying it.


Subscribe to my blog newsletter and download Continuous Digital for free

The post Scrum or OKRs, which comes first? appeared first on Allan Kelly.

The Excess Strategic WIP problem

Allan Kelly from Allan Kelly

Try pouring a bottle of milk into a glass with milk already in it. You have a choice: stop or tidy up the mess afterwards. That is my work-in-progress (WIP) analogy, if you try and do too much – no mater how much you want to do it – you will end up with a mess.

Agile folk are well versed in the problems created by too much WIP and how to deal with it – check out the Stockless Production video if you want to see. In the last six months I’ve been seeing a particular variant on this problem with I’ve come to label the Excess Strategic WIP problem.

In the latest report the manager told me how the team completed a great quarter with 3 priorities – set via OKRs. The senior management team were so impressed they asked for 19 priorities in the next period.

Right now I don’t have a “paint by numbers” solution, fixing this problem is more involved. I’m starting to understand it and I’ll make some suggestions later. Ultimately this a failure of leadership to say “No”. That failure is itself rooted in a failure of leadership to appreciate what is happening on the ground and what doing the worker are up against: the number of strategic initiatives or the amount of business as usual.

In a team it is easy to spot excess WIP using a visual system like a whiteboard or card system. When you track work like this you see an awful lot of work sitting in the “in progress” column. Typically there is more work than people on the team and the work isn’t moving. Some of the work may be actively marked as blocked but more likely most of it is nominally being worked on.

It can’t all be worked on at the same time because despite having two eyes, two hands, and two sides of a brain human’s can only really do one thing at a time – even new parents. I label this work “WHIP” – work hopefully in progress. While it is, in theory, being worked on, most of it is just sitting there waiting for a multi-tasking (i.e. time slicing) person to come back to it.

The good news is when you can see the WHIP you are half way to solving the problem: there are well known solutions. Accept less work, impose work in progress limits, sequence work by adding queues to the board, educate people to work on one thing until it is done, etc. Once you can see the work you have a feedback mechanism, you can take action and, thanks to the feedback mechanism, watch it reduce.

But Strategic WIP is more difficult. Strategic WIP is the stuff the organization decides in really important, the stuff the most senior leadership decides should be happening. The Excess Strategic WIP Problem occurs when those strategic priorities are greater than the organisation’s ability to deliver on them.

While some of this work may be transactional (“Build a Mega Widget”) much of it is transformational (“Adopt Agile working”, “Increase diversity”, “Tighten security”). Such things involve changing the way other work is done: changing the processes, changing the criteria, increasing awareness. The feedback cycle is long but to get started people need to devote time: attend training, arrange kick-off meetings, discuss approaches with consultants/coaches, etc.

In one case I saw this year the organization has, for years, been asked to do more than it is resourced to do. While a few months of such a mismatch might have been manageable the cumulative effect has been to create organizational debt and demoralised staff.

The second case I saw isn’t a result of under resourcing, if anything that organization has too much – money, people and perhaps equipment then they know what to do with. But because their management model resembles a sponge it is impossible to know when the organization has taken on too much. While some pockets were overworked I’m sure other pockets were idle.

In both these cases “lean” was a dirty word. Both organizations had been subject to lean programmes that had stripped out “waste” but, from what I could see, removing that waste had created both organizational debt and demoralised staff. Having removed “spare” staff those left were juggling. Staff didn’t have time for “agile” and their diaries had no space for daily essentials let alone another change programme.

The third case came to light when discussing OKRs. The company had a successful quarter were they focused on a few objectives that success seems to have bred the next failure when the organization requested many objectives.

Actually, this case brought it home: taking on too many, or being given too many, OKRs. I’ve heard of teams tackling too many OKRs so many times in the last year. (In fact, I should name this “Excess OKRs Problem”. This occurs whenever you have more OKRs than you can count on one hand. Don’t tell me your team is big enough to do so, check out Focus is not divisible so limit you OKRs.)

In a way, the Excess OKRs Problem is better than the Excess Strategic WIP Problem because you can see it. One can say “OK company, you have asked us to deliver 15 OKRs here, we need to sit down and talk about this.” Like visualising your work in progress excess OKRs is a thing you can identify and address, it is the reason to talk.

(Note: unlike User Stories which should always be small, OKRs should never be small. OKRs should be big, meaningful and preferably strategic. No team can take on 16 meaningful OKRs, even 5 is too many.)

One of the problems with excess Strategic WIP is that it can be difficult to see, there are different teams involved and in different places. Conflicting priorities are hidden. People on the ground may see problems but the senior people – the people creating the WIP – are too far removed. Those people may not want to hear people saying “You are asking too much”. They may have too much riding on getting multiple work streams done. They may be deaf to the cry of pain when people say “too much.” They may have too big an ego to accept that what they are asking for is a problem. And it may be politically unacceptable.

“Political” is an important word here: several of the cases I’ve heard of, and some of those example above, are Government agencies.

Because excess strategic WIP is difficult to see it is difficult to build a feedback loop and difficult to take action. How do you know when the problem is too much WHIP and when people are “crying wolf” ? – which in itself implies a problem of trust and maybe a belief that “everyone is lazy, we need to push harder.”

By its nature “strategy” is big, which means that the feedback cycles are long and the problems of excess strategic WIP take time to play out. What is a WIP problem looks like another failed strategy.

While I would like to think OKRs can help with this situation – because they force teams and organizations to take stock of what they are working on – they may be making things worse because OKRs include an ambition agenda. Teams are encouraged to “shoot for the moon” and build “10x solutions”. There is good logic here: if one aims for a “10x solution” (i.e. a solution 10 times better than the status quo) and falls short the “failure” may still be “better” (e.g. “5x”) than if one had aimed for a “2x solution” and succeeded.

Ambition with OKRs should not be about doing more OKRs, rather ambition is within the OKRs, a challenge that makes you approach it differently. One can draw a line here between having one OKR which aims for “10x” and having 10 OKRs but I suspect the subtlety will be lost on someone asking for “more than you think you can do.”

So whats is the solution?

I’m not sure there is a silver bullet but I would want to … make the problem visible, perhaps a portfolio level kanban board. I would want to build a feedback loop so I could measure change. I would want to show the strategic people the visualisation.

Management education has a part to play too. I can believe many senior managers would benefit from understand what WIP is, the problems excess WIP can cause, the way excess WIP plays out on a day-to-day basis and how it effects people’s working lives. And perhaps most of all, address trust and the belief that “we just need to push harder.”

That might also mean some of the Lean waste lessons and OKR ambition lessons need to be revisited.


Subscribe to my blog newsletter and download Continuous Digital for free

The post The Excess Strategic WIP problem appeared first on Allan Kelly.

Return of the sprint goal? (Infographic)

Allan Kelly from Allan Kelly

Most of the sprint goals I’ve ever seen are rubbish. Pretty much “do the random collection of stuff we’ve decided already.” Such goals are meaningless – save time by skipping them.

If a team adopts OKRs then I really hope they move towards more goal based working, in which case the sprint goal starts to have meaning again – although maybe the OKR replaces the sprint goal?

Either way, I see an opportunity to move away from backlog driven development (BLDD) and towards a more purposeful style of working. So it was interesting a few weeks ago when Gareth Davies from Parabol (online meeting exercises and such) sent me over this infographic and a link to a blog on Sprint goals. Food for thought.


Subscribe to my blog newsletter and download Continuous Digital for free

The post Return of the sprint goal? (Infographic) appeared first on Allan Kelly.