I am sure some of you have noticed that my blogs have been a less regular the last few months. That is because I’ve been busy on other stuff. So a break from deep thoughts and advice on the software world to mention some other stuff I’ve been working on.
To my surprise Little Book has long been my best-seller so I teamed up again with Stacy Gonzalez – who voiced Project Myopia for me – to produce an audio version of Little Book. In the few weeks it has been available sales are already outstripping Project Myopia!
(And if you can’t wait, I’ve got a pre-copy edit version I can share with you provided you promise to write an Amazon review when the book is published. Mail me or use the contact form if you are interested.)
Finally, that picture at the top of the page. I’ve been working with Nicolas Umiastowski to create a playing card retrospective. These are based on my Retrospective Dialogue Sheets In our experiments they have given retrospectives another twist. More about these soon – and details of how you can get a pack (in the mean time get in contact if you are really keen to try them.)
Like this post? – Like to receive these posts by e-mail?
Thanks to Darwin, the world is full of people who think that evolution, in nature, works by: natural selection, or the survival of the fittest. I thought this until I read “Good Enough: The Tolerance for Mediocrity in Nature and Society” by Daniel Milo.
Milo makes a very convincing case that nature actually works by: natural elimination, or the survival of the good enough.
Why might Darwin have gone with natural selection in his book, On the Origin of Species? Milo makes the point that the only real evidence that Darwin had to work with was artificial selection, that is the breeding of farm animals and domestic pets to select for traits that humans found desirable. Darwin’s visit to the Galápagos islands triggered a way of thinking, it did not provide him with the evidence he needed; Darwin’s Finches have become a commonly cited example of natural selection at work, but while Darwin made the observations it was not until 80 years later that somebody else spotted their relevance.
The Origin of Species, or to use its full title: “On the Origin of Species by means of natural selection, or the preservation of favored races in the struggle for life.” is full of examples and terminology relating to artificial selection.
Natural selection, or natural elimination, isn’t the result the same?
Natural selection implies an optimization process, e.g., breeders selecting for a strain of cows that produce the most milk.
Natural elimination is a good enough process, i.e., a creature needs a collection of traits that are good enough for them to create the next generation.
A long-standing problem with natural selection is that it fails to explain the diversity present in a natural population of some breed of animal (there is very little diversity in each breed of farm animal, they have been optimized for consistency). Diversity is not a problem for natural elimination, which does not reduce differences in its search for fitness.
The diversity produced as a consequence of natural elimination creates a population containing many neutral traits (i.e., characteristics that have no positive or negative impact on continuing survival). When a significant change in the environment occurs, one or more of the neutral traits may suddenly have positive or negative survival consequences; the creatures with the positive traits have opportunity time to adapt to the changed environment. A population whose members possess a diverse range of neutral traits has a higher chance of long-term survival than a population where diversity has been squeezed in the quest for the fittest.
I think that natural elimination also applies within software ecosystems. Commercial products survive if enough customers buy them, software developers need good enough know-how to get the job done.
I’m sure customers would prefer software ecosystems to operate on the principle of survival of the fittest (it reduces their costs). Over the long term is society best served by diverse software ecosystems or softwaremonocultures? Diversity is a way of encouraging competition, but over time there is diminishing returns on the improvements.
Last month we saw how we could efficiently generate hierarchical clusterings, which are sequences of sets of clusters, which are themselves subsets of a set of data that each contain elements that are similar to each other, such that if a pair of data are in the same clustering at one step then they must be in the same clustering in the next which will always be the case if we move from one step to the next by merging the closest pairs of clusters. Specifically, we used our ak.minHeap implementation of the min-heap structure to cache the distances between clusters, saving us the expense of recalculating them for clusters that don't change from one step in the hierarchy to the next.
Recall that we used three different schemes for calculating the distance between a pair of clusters, the average distance between their members, known as average linkage, the distance between their closest members, known as single linkage, and the distance between their farthest members, known as complete linkage, and that I concluded by noting that our algorithm was about as efficient as possible in general but that there is a much more efficient scheme for single linkage clusterings; efficient enough that sorting the clusters in each clustering by size would be the most costly operation and so in this post we shall implement objects to represent clusterings that don't do that.
Some years ago I was managing a team at an internet TV pioneer. Shortly after a release our biggest customer – ITV – was on the phone complaining about a bug. Unfortunately one set of data fields were retaining their value when they should be wiped clear. It took a couple of months before we could include a fix in the release but we were confident we would make ITV happy.
A couple of hours after the fix went live ITV were on the phone. Why had we removed the daily cache? They weren’t just “working around†the bug, they were utilising it as a feature.
Sometimes it seems the problem you think you are dealing with isn’t the problem you are dealing with. Sometimes the way to solve a problem is not to fix the problem head on. Rather the solution comes from reframing the problem so that it is easier to solve. To put it another way: the problem you are trying to solve is knowing the problem you are trying to solve.
I sometimes think of this as “go and look at it from a different placeâ€. Whatever you are looking at – problem, opportunity, objective, way of life – looks different if you go and stand somewhere else. The more different the place you stand in the more different the thing looks.
Clearly one way of solving a problem is to define it as not a problem: “it is not a bug, it is a feature.†As the story above shows, one person’s “bug†can be another person’s “feature.â€
The question is then: is the reframing acceptable to others? – can others share the reframed perspective?
By way of example, lets apply this to agile.
A big part of agile is focus: small user stories, morning stand-ups and sprint planning all help create focus. Once you decide what you are going to tackle you focus on it, push other things out of the way and do it. And do it to completion.
You might call this “eating the elephantâ€: don’t eat it all at once, carve off a small piece, eat and repeat.
Pushed to either extreme this approach has bad side effects. Focus too tightly or too inflexibility and you might deliver a thing but not a thing which others recognise as a solution to the problem. But taking a view too broadly, or being very flexible, negates the way of working because you don’t deliver anything, – or the solution you deliver doesn’t please anyone (the “you can’t please all the people all the time†problem.)
Reframing can be a powerful technique but it can also work against you if other’s reframe the problem.
So, how do you know, or rather how do you frame and decide what your objective is?
Deciding what the problem is requires some imagination, it requires focus and flexibility – to stare at the problem but be flexible in how you see it. It also requires understanding and then the ability to communicate that understanding to others.
While I often hear people say “we should focus on the problem†I wish we could spend more time focusing on the problem of knowing what the problem is.
Reframing the problem is an inherently human thing. While machines may now, or in the near future, be able to solve many problems there is a problem of knowing what problem to present to the machine.
A big part of human work – often managerial work – is framing problems and deciding which to solve and which not to solve. Framing can make a small problem big, or a big problem small. That is inherently a human activity.
So, how do you know what problems to address? how do you know where to look for opportunities?
Now isn’t that a problem in itself? – surely we could set that up as an objective in its own right. Again it needs defining and framing and… So how does one decide to eat an elephant? And which elephant? And even, why eat an elephant at all?
It all becomes recursive. You can’t recurse for ever so hopefully sooner or later you need to come to the top – otherwise you have just reinvented paralysis by analysis. Detailed problem thinking needs to be combined with vaguer, even oblique, thinking.
To make this very real: I’m a big fan of metrics, they help focus, they help you know if you are going in the right direction. But I detest metrics too because they are a blunt instrument which can mislead so easily.
Metrics are great for focus but they need to be combined with a healthy dose of scepticism and oblique thinking. Metrics have limitations.
Sorry no solutions today. Just awareness of the problem of problems.
Like this post? – Like to receive these posts by e-mail?
