In the previous post we saw how to identify subsets of a set of data that are in some sense similar to each other, known as clusters, by constructing sequences of clusterings starting with each datum in its own cluster and ending with all of the data in the same cluster, subject to the constraint that if a pair of data are in the same cluster in one clustering then they must also be in the same cluster in the next, which are known as hierarchical clusterings.
We did this by selecting the closest pairs of clusters in one clustering and merging them to create the next, using one of three different measures of the distance between a pair of clusters; the average distance between their members, the distance between their nearest members and the distance between their farthest members, known as average linkage, single linkage and complete linkage respectively.
Unfortunately our implementation came in at a rather costly O(n3) operations and so in this post we shall look at how we can improve its performance.
Those of us who donâ€™t code any more, and perhaps many of those who do, need Electronic Monks to help us with software development.
There is an old Douglas Adamâ€™s book (Dirk Gentlyâ€™s Holistic Detective Agency) which features an Electric Monk. The job of an electric monk is to believe things for you. In Adamâ€™s story people have too many things to believe so they offload all that believing to an electric monk. In my mind Iâ€™ve always considered part of the monkâ€™s work to include worrying. I canâ€™t remember if Adamâ€™s says this explicitly or just implies it. (And as I no longer have a copy of the book I canâ€™t check.)
Iâ€™m currently very engaged with one client as an Agile Coach (although I sometimes wonder if â€œShadow Managerâ€ might be a better term, more of that another day). I regularly find myself staring at the board thinking about the work it shows and worrying about whether it will be done. Sometimes my mind plays â€œwhat if gamesâ€ – â€œIf that card is finished soon, then the other one could move down andâ€¦â€
The same worry plays out when looking at the backlog showing work not on the board. Or when Iâ€™m talking to the Product Owner. Or indeed when I find myself talking to middle managers and others in the company who have an interest in the work the team is doing.
Basically, there is very little any of us can do to move the work through.
Sure I can call a meeting and talk about optimising our workflow. But Iâ€™ve done that a few times already.
I could call a meeting and emphasis how important the work is to the company. Iâ€™ve seen this done many times. Some non-coders – call them â€œthe businessâ€ – seem to think â€œIf only the coders appreciated how important this work was then they would do it faster.â€ I hated this when I was a coder and I hate it when I hear others saying things like â€œDo they know how important this is?â€ Business folk sometimes seem to believe coders sit around drinking tea and coffee for most of the day.
By the way, I almost wrote â€œand Testersâ€ in that last paragraph but then realised testers CAN make work go faster: they can just drop their standards, turn a blind eye to issues, accept things they wouldnâ€™t have accepted last week. Piling pressure on Testers is a more effective route to getting work done than pressuring coders. But in both cases it is probably just piling up more work.
I could go to the coders and ask â€œAre you done yet?â€ – or as 5-year olds say in the back of a car on a long journey (or just any journey) â€œAre we there yet?â€ In both cases it doesnâ€™t change the time it takes to get to the end, the answer doesnâ€™t usually say much but asking the question will annoy parents and coders alike.
But if I do anything – like calling a meeting or asking â€œare we there yet?â€ – which involves distracting coders from working then I am slowing work down. It is self-defeating.
Yes there are things I can do to make work go more smoothly but once a piece of work is in flight there is little I can do. Most of the change I can make are to do with the way work happens. Or to put it another way: I can influence the climate work happens in but I canâ€™t control the individual weather events which are the work items.
All I can do is worry. Thus my desire to offload that worrying to an electronic monk. Maybe more people in and around software developerment need to recognise they are in the same position.
Like this post? – Like to receive these posts by e-mail?