Posted on

Pandemic in the digital age

Allan Kelly from Allan Kelly Associates

plastic-syringe-and-test-tubes-2020-04-14-10-10.jpg

It was hoping to keep this blog virus free. Indeed my “Conflicts in coaching” was going to be the first of several on agile coaching (what else could I do in the air going to and from Agile on the Beach New Zealand?) But…. the world has changed, I’ve changed…

It is a very scary time. Both health wise and economically: I know at least one software engineer who has lost his job as a result of the slow down. But I also know random (inappropriate) coding jobs still appear in my mailbox, I continue to see job adverts on Twitter and LinkedIn and I know one company that has landed work and had to hired contractors to work on a corvid-19 project. So some observations…

Observation 1: Covid-19 will go down in history as the first digital health crisis.

Digital technology has a big role in fighting the virus. Decisions and actions are being driven by software models of what could happen. The famous Imperial model is now OpenSource and Microsoft engineers are reported working to improve the model. (At a few hundred lines of R code there isn’t that much to refactor – although there are some very long functions and I can’t see any unit tests.)

Apps are being created to track contacts and you can bet that the search for antidotes and vaccines is utterly dependent on software. Digital powered home delivery networks and internet shopping have made closing the economy just about possible.

Those who are not directly fighting the virus are continuing to work because of digital technology. Zoom, Skype, and the like might be the most obvious beneficiaries of the virus but many others will benefit too. Although the virus is simultaneously putting a strain on our digital infrastructure and necessitating human action – witness the search for Cobol programmers in the US.

Not only have most IT, sorry digital, workers decamped to home but so too have many others – in fact almost any occupation that can. Schools are delivering lessons and distributing home learning kits online. Industries which can’t move to online working will suffer the most. (Except those which put themselves in harms way like medical staff and, to a lesser degree, delivery staff.)

And when not working online media like Netflix, YouTube and BBC iPlayer keep us sane.

For us digital folk this is no big deal. It is an extension of normal life: we are at home 5 days a week not one. But for other folk, this is big. Even the most digitally inept lawyer is having to get with the technology. As people are forced to become familiar with digital technology …

Observation 2: Digital technology adoption will be accelerated by the virus

Which means, while some technology companies (like my friend’s) will not survive, those that do are set for a boom. Post virus swaths of the economy will be destroyed but technology is in for a boom.

That boom is driven by the three forces above: 1) unlike others, our industry is not destroyed, 2) technologist continue to work remotely, and 3) non-technologist will learn to use more technology.

In particular digital healthcare – both back-office big data background analysis and customer centred applications – will play an oversized part. This field was already growing rapidly but the experience gained during this crisis can only help the sector.

But also…

Observation 3: The economic devastation caused by the virus will open up many new opportunities for digital companies to enter markets and thrive

Companies which fail create opportunities for new companies – either a like-for-like replacement or a new type of company. Previously, while those companies were active, digital technology had to compete with the existing providers, the incumbents. With those companies gone the way is clear for new digital technology companies to enter the market.

I’m not saying this isn’t going to be horrible; company failures will be painful and it new entrants will take time to get established.

And what of Agile?

Observation 4: Covid-19 is the ultimate test of agility

Forget arguments about what is agile and what is not agile. Forget ScrumBut, Wagile and the other insults hurled at those judged to be less agile than thou.

Forget agile assessments and agile maturity frameworks; forget ticking off ceremonies and declaring yourself agile. In the new world the more agile you are the greater your chances of survival.

On paper you may have the most agile team in the world but, if that team, and your organization, cannot now demonstrate how it changes rapidly it just isn’t agile.

Every single plan that existed before March 1st is now invalid. Right now companies need to pivot like never before. Agility helps companies pivot. Those who can’t pivot, or can’t pivot fast enough stand to loose the most. If you can’t pivot you aren’t agile, QED.

Companies which still operate in hierarchal command-and-control mode will find it more difficult to switch to distributed teams and remote working. When everyone is remote you need to delegate decision making. Companies which don’t trust employees, companies which constantly check what employees are doing will find home working incredibly difficult and expensive.

Individuals and interactions are more important than ever before. Processes and tools are essential but few heavy weight processes will survive the instant shift to completely distributed working. Any tool which doesn’t help now is an impediment.

Those companies which are still struggling with technical liabilities (aka technical debt) will find the cost of living with those liabilities just increased.

Observation 5: Test driven medicine

Day after day I read in the papers that the UK is not doing enough testing. It seems that countries like South Korea which do a lot of tests and base their strategy on knowing who is infected (and therefore who is safe) and then tracing the virus are doing best.

That means testing needs to be rapid – a short feedback loop.

And testing needs to be cheap so it can be done at scale.

Doesn’t that sound familiar?

The cost of not testing is precautionary isolation. That cost is not sustainable.

If you could test anyone, and everyone, instantly the offices, shops and schools could reopen: you would just test everyone who arrives.

The testing strategy agile has been advocating is now needed to fix the world. And in the UK the Government seems to be as resistant to a test first approach as the most obstinate software manager or engineer.

As much as I hope the world will shortly return to how it was it will not. It will never be the same, we don’t quite know how it will be but it is already clear that digital technology and agility will be part of it.

(Test tube image taken from PublicDomainPctures.net)

Subscribe to my blog newsletter and download Project Myopia for Free

The post Pandemic in the digital age appeared first on Allan Kelly Associates.

Posted on

Allan’s online offerings

Allan Kelly from Allan Kelly Associates

iStock-1213772070lite-2020-04-13-15-52.jpg

I admit it: I’ve been scared of running online training workshops for years.

I just didn’t see how they fitted with the way I liked doing things. Online training I’ve joined in the past has been boring and I’ve found it difficult to keep my attention. So I assumed that everyone was like that and all online training suffered from the same failings.

But like so many other people in the last few weeks I’ve been forced to reconsider.

And I’ve had really positive experiences. In fact, I’m wondering if online might actually be better suited to my preference for interaction and discussions. Although I’ll admit, it makes designing exercises harder.

Last week I delivered what was a 2-day agile course as four half-day workshops. This seems like a win-win: it is less disruptive for the team undertaking the course, avoids days spent on video meetings and provides more opportunities to adjust content to suit need – plus its cheaper!

So, as of now I have two workshops redesigned and tested online. In addition I’m making myself available for online consulting/coaching. In the past it was only feasible to provide consulting and coaching to companies who could buy whole days. Delivering this online makes it possible to work with individuals.

Improving with Agile
Based on my existing and much delivered Agile Foundations course this four half-day online workshop looks at how teams can use ideas from the agile toolkit to work better. This could mean going from “no agile” to “agile”, or from “existing agile” to “better agile.”

Stories and Value
Derived from my long standing Requirements, Backlogs and User Stories 1-day workshop this is now delivered as four 90-minute workshops. This revised version continues to look at how to write good user stories however now the emphasis is on understanding and delivering business benefit, i.e. value.

I’m part way through delivering Stories and Value to those who signed up to my free workshop offer a few weeks back.

These are on my website now.

I may well offer both of these workshops as public (open enrolment) courses in the next few weeks – although this time I will need to levy a charge. If history is a guide I expect private deliveries to existing teams will be the main customers.

If you are interested in either – joining a public workshop or having this workshop for your own team – please contact me. Who’s interested?

I’m still finalising my plans for personal coaching and consulting, so again, if you are interesting in getting some of my time please get in touch.

And if you have suggestions for workshops or tutorials I should offer I’d love to hear your ideas!

Subscribe to my blog newsletter and download Project Myopia for Free

The post Allan’s online offerings appeared first on Allan Kelly Associates.

Posted on

Motzkin paths and source code silhouettes

Derek Jones from The Shape of Code

Consider a language that just contains assignments and if-statements (no else arm). Nesting level could be used to visualize programs written in such a language; an if represented by an Up step, an assignment by a Level step, and the if-terminator (e.g., the } token) by a Down step. Silhouettes for the nine possible four line programs are shown in the figure below (image courtesy of Wikipedia). I use the term silhouette because the obvious terms (e.g., path and trace) have other common usage meanings.

Number of distinct silhouettes for a function containing four statements

How many silhouettes are possible, for a function containing n statements? Motzkin numbers provide the answer; the number of silhouettes for functions containing from zero to 20 statements is: 1, 1, 2, 4, 9, 21, 51, 127, 323, 835, 2188, 5798, 15511, 41835, 113634, 310572, 853467, 2356779, 6536382, 18199284, 50852019. The recurrence relation for Motzkin numbers is (where n is the total number of steps, i.e., statements):

(n+2)m_n = (2n+1)m_{n-1}+3(n-1)m_{n-2}

Human written code contains recurring patterns; the probability of encountering an if-statement, when reading code, is around 17% (at least for the C source of some desktop applications). What does an upward probability of 17% do to the Motzkin recurrence relation? For many years I have been keeping my eyes open for possible answers (solving the number theory involved is well above my mathematics pay grade). A few days ago I discovered weighted Motzkin paths.

A weighted Motzkin path is one where the Up, Level and Down steps each have distinct weights. The recurrence relationship for weighted Motzkin paths is expressed in terms of number of colored steps, where: l is the number of possible colors for the Level steps, and d is the number of possible colors for Down steps; Up steps are assumed to have a single color:

(n+2)m_n = d(2n+1)m_{n-1}+(4c-d^2)(n-1)m_{n-2}

setting: c=1 and d=1 (i.e., all kinds of step have one color) recovers the original relation.

The different colored Level steps might be interpreted as different kinds of non-nesting statement sequences, and the different colored Down steps might be interpreted as different ways of decreasing nesting by one (e.g., a goto statement).

The connection between weighted Motzkin paths and probability is that the colors can be treated as weights that add up to 1. Searching on “weighted Motzkin” returns the kind of information I had been looking for; it seems that researchers in other fields had already discovered weighted Motzkin paths, and produced some interesting results.

If an automatic source code generator outputs the start of an if statement (i.e., an Up step) with probability a, an assignment (i.e., a Level step) with probability b, and terminates the if (i.e., a Down step) with probability c, what is the probability that the function will contain at least n-1 statements? The answer, courtesy of applying Motzkin paths in research into clone cell distributions, is:

P_n=sum{i=0}{delim{[}{(n-2)/2}{]}}(matrix{2}{1}{{n-2} {2i}})C_{2i}a^{i}b^{n-2-2i}c^{i+1}

where: C_{2i} is the 2i‘th Catalan number, and that [...] is a truncation; code for an implementation in R.

In human written code we know that a != c, because the number of statements in a compound-statement roughly has an exponential distribution (at least in C).

There has been some work looking at the number of peaks in a Motzkin path, with one formula for the total number of peaks in all Motzkin paths of length n. A formula for the number of paths of length n, having k peaks, would be interesting.

Motzkin numbers have been extended to what is called higher-rank, where Up steps and Level steps can be greater than one. There are statements that can reduce nesting level by more than one, e.g., breaking out of loops, but no constructs increase nesting by more than one (that I can think of). Perhaps the rather complicated relationship can be adapted to greater Down steps.

Other kinds of statements can increase nesting level, e.g., for-statements and while-statements. I have not yet spotted any papers dealing with the case where an Up step eventually has a corresponding Down step at the appropriate nesting level (needed to handle different kinds of nest-creating constructs). Pointers welcome. A related problem is handling if-statements containing else arms, here there is an associated increase in nesting.

What characteristics does human written code have that results in it having particular kinds of silhouettes? I have been thinking about this for a while, but have no good answers.

If you spot any Motzkin related papers that you think could be applied to source code analysis, please let me know.

Posted on

Life, the Universe and Everything

Jon Jagger from less code, more software

is an excellent book by Douglas Adams (isbn 978-0-330-49120-4). As usual I'm going to quote from a few pages.

He would insult the universe. That is, he would insult everybody in it. Individually, personally, one by one, and (this was the thing that he really decided to grit his teeth over) in alphabetical order.
'Eddies,' said Ford, 'in the space-time continuum.'
'Ah, nodded Arthur, 'is he? Is he?'
It was a charming and delightful day at Lord's as Ford and Arthur tumbled haphazardly out of a space-time anomaly and the immaculate turf rather hard.
'A what?' he said.
'An SEP.'
'An S...?'
'...EP.'
'And what's that?'
'Somebody Else's Problem,' said Ford.
The second non-absolute number is the given time of arrival, which is now known to be one those most bizarre of concepts, a recipriversexcluson, a number whose existence can only be defined as being anything other than itself. In other words, the given time of arrival is the one moment of time at which it is impossible that any member of the party will arrive.
'Tell me it was a coincidence, Dent' it said. 'I dare you to tell me it was a coincidence!' 'It was a coincidence,' said Arthur quickly.
It was the Cathedral of Hate.
It was the product of a min that was not merely twisted, but actually sprained.
It was huge. It was horrific.
It had a Statue in it.
We will come to the Statue in a moment.
'I got yanked involuntarily back into the physical word,' pursued Agrajag, 'as a bunch of petunias. In, I might add, a bowl.'
Strictly speaking, all editors since Lig Lury Jr have therefore been designated Acting Editors, and Lig's desk is still preserved the way he left it, with the addition of a small sign which says 'Lig Lury Jr, Editor, Missing, presumed Fed'.
The longest and most destructive party ever held is now into its fourth generation and still no one shows any signs of leaving.


Posted on

Comments on the COVID-19 model source code from Imperial

Derek Jones from The Shape of Code

At the end of March a paper modelling the impact of various scenarios on the spread of COVID-19 infections, by the MRC Centre for Global Infectious Disease Analysis at Imperial College, appears to have influenced the policy of the powers that be. This group recently started publishing their modelling code on Github (good for them).

Most of my professional life has been spent analyzing other peoples’ code, for one reason or another (mostly Fortran, then Pascal, and then C). I had heard that the Imperial software was written in C, but the released code is written in R (as of six hours ago there is the start of a Python version). Ok, I can work with R, but my comments will be general, since I don’t have lots of in depth experience reading R code.

The code comes from a research context, and is evolving, i.e., some amount of messiness is to be expected.

There is not a lot of code to talk about (248 lines setting things up, 111 lines for a Stan model, 371 lines of plotting code, and 85 lines of utility code). The analysis is performed by creating a model using the Stan statistical inference language (in which the high level structure of the problem is specified, compiled to a lower level form and then run; the Stan language is very similar to R). These days lots of problems are coded using a relatively small number of lines that call fancy libraries to do the heavy lifting. It is becoming rare to have to write tens of thousands of lines of code to solve a problem.

I have two points to make about the code, all designed to reduce the likelihood of mistakes being made by the person working on the source. These points mainly apply to the Stan code, because that is where the important stuff happens, but are equally applicable to all code.

  • Numeric literals are embedded in the code, values include: 2.4, 1.0, 0.5, 0.03, 1e-5, and 1e-9. These values obviously mean something to the person who wrote the code, and they can probably be interpreted by experts in the spread of virus infections. But why are they scattered about the code, rather than appearing together (as a sequence of assignments to variables with meaningful names)? Having all the constants in one place makes it easier to spot when a mistake has been made, e.g., one value has been changed without a corresponding change in another value; it also makes it easier for people new to the code to figure out what is going on,
  • when commenting out code, make it very obvious, e.g., have /********************** on its own line, and *****************************/ on its own line. Using just /* and */ makes it easy to miss that code has been commented out.

Why have they started a Python implementation? Perhaps somebody on the team is more comfortable working with Python (when deadlines loom, it is always best to go with what you know).

Having both an R and Python version is good, in that coding mistakes are likely to show up as inconsistencies in the results produced. It’s always good to have the output of two independently written programs to compare (apart from the fact it may cost twice as much).

The README mentions performance issues. I imagine that most of the execution time is spent in the Stan code, so R vs. Python is not a performance issue.

Any reader with expertise tuning Stan models for performance might like to check out the code. I’m sure the Imperial folk would be happy to hear about worthwhile speed-ups.

Posted on

Keeping track of podcast times with a simple bash script on Linux

Andy Balaam from Andy Balaam's Blog

I was recording some podcast audio tonight and wanted to be able to press a single key when I reached a significant moment, so I could add the times to the show notes.

I couldn’t find anything that already did this, so I wrote a tiny bash script. I ran this script pressed Enter whenever I wanted a time recorded:

T=0
echo
while sleep 1; do
    echo -n -e "\e[1A"
    echo $(($T / 60))m $(($T % 60))s
    T=$(($T + 1))
done
Posted on

All Your Basis Are Belong To Us – a.k.

a.k. from thus spake a.k.

A few years ago we saw how we could approximate a function f between pairs of points (xi, f(xi)) and (xi+1, f(xi+1)) by linear and cubic spline interpolation which connect them with straight lines and cubic polynomials respectively, the latter of which yield smooth curves at the cost of somewhat arbitrary choices about their exact shapes.
An alternative approach is to construct a single function that passes through all of the points and, given that nth order polynomials are uniquely defined by n+1 values at distinct xi, it's tempting to use them.
Posted on

Coaching conflicts

Allan Kelly from Allan Kelly Associates

CoachingConflicts-2020-04-1-12-36.jpg

Last year I was coaching a team. One of the big problems the team faced was excessive work in progress – and tendency for developers to start new work when they hit a blockage. Eventually, with the help of the Product Owner who saw the problem too, we starved the work pipeline. The team actually ran out of work. We saw this as a great success. It had never happened before and meant we could really focus and prioritise work.

Unfortunately, this happened when the two of us were not instantly available. You can argue that we should have been instantly available. Or that people should have made more of an effort to contact us. Or that we should have left a secret stash of work to do. Or that the team should have self-organized to fix the problem. That is easy in retrospect but really, I still don’t see it as a problem.

A few hours without work? I see it as a momentous moment, the start of something great.

But that is not how others saw it. The team, the Product Manager, another Agile Coach on site, and anyone these people could tell were quick to tell us how awful it was: “the team ran out of work.” Word spread quickly that the team had run out of work. My name was dirt.

Doing good by one group isn’t always seen as good by others. When you work as an agile coach conflicts occur daily. But some are bigger and more persistent than others.

For most of the last 10 years I’ve mainly worked as a “drop in” coach (“light touch” I like to call it). I visit clients for a short period of time, talk about improvements, problems, solutions, give directive or non-directive coaching and don’t return for days or weeks. I’m not the same clients every week, maybe I drop in a few days a month and talk to people.

Last year was different, I spent almost all of it with one company, mostly embedded in one team as an official “Agile Coach.”

Comparing these two approaches to coaching has left me with a lot to reflect on. In particular I found a number of conflicts which troubled me, I’m not sure I ever worked these out and I’m sure others find the same things. So I’d like to share…

Responsible to the team, accountable to the organization

I was lucky, it was one of the team members who called me in but it was the bigger organization that was paying my fees. While I felt responsible to the team I was accountable to the organization.

That organization wanted things and expected me to deliver: they wanted a team which performed better (delivered more stuff and more value!) They wanted the team to share common practices and ceremonies with other teams.

For the organization I was the bringer of change to the team. But I could only do that if I was accepted by the team. That limited my ability to push through changes. Even if nobody else saw that conflict I felt it every day.

At one level team did want to change but they also wanted the organization to change and I had very little power there. Both sides, the team and the organization had no-go areas. More conflict.

The organization restricted my ability to do things I thought would improve the team. Things the team accepted would help: like spending money on training. So I was bearer of bad news to both sides: one side saw me asking, then arguing for money while the other saw me failing to deliver.

That organization also expected me to operate within the organization: join coaches meetings, sign up to shared coaching goals, complete team assessments, etc. None of these things were necessarily bad in their own right but it meant I had two masters: the team and the organization.

When push came to shove I prioritised the team but I know some coaches who prioritised the organization. I know some team members mistrusted their coaches because they believed their coach would put the organization first.

Honouring self-organization but creating change

So an agile team is self-organizing. That gives them the right to self-organise to work exactly as they do today. Self-organization gives them the right to not change anything – something I wrote about way back in Changing Software Development.

But, almost by definition, the (agile) coaches role is to bring about change, to help the team do better. Conflict is inevitable.

Sure you say “its a question of motivation … the coach needs to create the motivation to change and do better” and I would agree, but, even in creating that motivation one is creating change, one is intervening. Which brings us nicely to….

Leading without authority

Agile coaches lack authority – if they had authority they would be managers, I’ll blog about that in future. In a way not having authority is liberating, one can’t use the whip no matter how much one wants to! But it is also difficult.

The organization, and the coach, wants to create change but without authority even the smallest changes can become massive efforts. When the team is divided themselves, or even when one team member objects to implementing a change becomes like wading through treacle. That can be demoralising for the coach.

Yet a little authority can go a long way in pushing through change and overriding objections.

And on occasions I did reach for authority, but that creates a conflict within oneself as a coach: was I right to do it? am I honouring the team? the team members? am I creating a learned dependency?

Accepted while pushing the unpopular

Nowhere is that conflict clearer then when pushing through an unpopular change in the face of opposition – even minority opposition. As a coach one risks loosing future changes because, most change the coach “initiates” is done with the acceptance of the team, pushing through an unpopular change – even with a majority, even with leadership support – risks future acceptance.

One is constantly asking: how far can I take this team right now?

And: if I take them too far will they trust me tomorrow?

And, most of all: am I right to do this?

Hardly a day pasted last year when I didn’t agonise over these questions. And as I write this I imagine one of those teams members reading this and saying “Huh, and you got it wrong.”

Who gets the credit?

As a coach your job is to make others perform better, but really, only they can perform better. You can’t make them, you can only help them. The final decisions rest with them.

So who should get the credit? – surely it is them, they made the change, they did something different.

That creates an inner conflict. It also creates a conflict with the organization: why should they keep me employed? After all I didn’t make any difference, they did it.

We know the value of positive praise and acknowledgement, but when there is nobody to praise you, when the team don’t recognise the coaches role (which can be hard if the coach is doing a good job) then one becomes demoralised and that saps ones strength to carry on.

As people we need acknowledgement, as a human we all have needs. But the coaches role so often demands that we forego acknowledgement, praise and recognition.

Conflicts exists

This isn’t an exhaustive list of the conflicts I’ve encountered and hopefully as you read this you can see solutions – I can myself! But what I want to say is: these conflicts exist, I’m sure other coaches have them and even when there are solutions those solutions need to be applied.

Living with these conflicts can be hard, mentally and emotionally. Burnout happens to coaches.

And organizations get fed up with coaches who don’t deliver change, don’t turn up to non-team meetings, keep asking for money, don’t crack the whip or exceeds their (none existent) authority.

Subscribe to my blog newsletter and download Project Myopia for Free

The post Coaching conflicts appeared first on Allan Kelly Associates.