No unified theory of agile (Agile mindset cont.)

Allan Kelly from Allan Kelly

Continuing my quest of “The Agile Mindset” I’ve been searching for a metaphor to put all the different ideas on agile into order. To cut to the chase: there isn’t one.

As much as I would love to boil “agile” down to one thing, or even a handful of key concepts and I don’t think there will ever be a single unified theory of agile. As I said before with the elephant example, everyone sees something different. Depending on where you are standing, the problems you face today, your own history and area of knowledge, and your own world view you are going to see and emphasise different aspects of “The Agile Mindset.” And you know what? that is a good thing!

First I tried thinking of Agile as the layers of an onion. The outside skin is the word “agile” – there is a valid reason for saying the agile mindset is there in the dictionary definition of the word agile: able to move, think and understand quickly, be nimble and subtle in movements, alert and observant when looking at whats happening and sharp when thinking. But that is only the outside layer, it is very general.

The agile manifesto would be the second layer. While the manifesto is specific to software it doesn’t require too much thought to generalise it to other domains. Actually, it is a bit too easy and there are more attempts to generalise it than there are people who have attempted to generalise it. Plus, as I’ve written before, the manifesto is over 20 years old, those who cling to it sound like Supreme Court Justices trying to read meaning into a document written in a different age.

And what are the next layers, and in what order? Does last responsible moment come above or below cost of delay? Is test driven more important than time-boxing? Are work-in-progress limits a version of time-boxing or an alternative to time-boxing?

And what is at the centre? For me it is learning but I can imagine people who will say it is People – perhaps manifested through Weinberg’s “Its always a people problem” quote – I’ve written about that one too, the People Problem Problem. Personally I think McGregor’s Theory X and Theory Y could be a candidate, which raises the question of agile’s fellow travellers – beyond budgeting, system thinking, Lean, and Mintzberg’s theory of emergent strategy.

I wondered if agile could be thought of as a brick wall, with each idea forming a brick, and then the whole being more than the sum of the parts. But that falls down (sorry for the pun!) on the layering problem. Which ideas are foundations and which decorative?

Similarly, I toyed with The House of Agile with different ideas represented by different rooms but that metaphor quickly runs into problems too.

In a way this makes the search fo One Agile Mindset even more desirable – the search for a grand unified theory of everything if you like. There must be something out there that combines all of this!

Ye the search for the unifying theory also highlights how damn difficult this is. Intellectually it is hard to accept that “the agile mindset” is a bunch of different ideas which different people interpret differently.

But you know what? Accepting that agile is diverse is itself agile – agile is not one idea, it is many, accepting that and valuing those different ideas mean embracing diversity and that itself is agile. Agile is what you want it to be because through those diverse we find alternative ways of viewing and learning. Agile doesn’t stand still, agile is punk, at its best agile is democracy.

Unfortunately that makes it hard to explain.


Subscribe to my blog newsletter and download Continuous Digital for free

The post No unified theory of agile (Agile mindset cont.) appeared first on Allan Kelly.

Anthropological studies of software engineering

Derek Jones from The Shape of Code

Anthropology is the study of humans, and as such it is the top level research domain for many of the human activities involved in software engineering. What has been discovered by the handful of anthropologists who have spent time researching the tiny percentage of humans involved in writing software?

A common ‘discovery’ is that developers don’t appear to be doing what academics in computing departments claim they do; hardly news to those working in industry.

The main subfields relevant to software are probably: cultural anthropology and social anthropology (in the US these are combined under the name sociocultural anthropology), plus linguistic anthropology (how language influences social life and shapes communication). There is also historical anthropology, which is technically what historians of computing do.

For convenience, I’m labelling anybody working in an area covered by anthropology as an anthropologist.

I don’t recommend reading any anthropology papers unless you plan to invest a lot of time in some subfield. While I have read lots of software engineering papers, anthropologist’s papers on this topic are often incomprehensible to me. These papers might best be described as anthropology speak interspersed with software related terms.

Anthropologists write books, and some of them are very readable to a more general audience.

The Art of Being Human: A Textbook for Cultural Anthropology by Wesch is a beginner’s introduction to its subject.

Ethnography, which explores cultural phenomena from the point of view of the subject of the study, is probably the most approachable anthropological research. Ethnographers spend many months living with a remote tribe, community, or nowadays a software development company, and then write-up their findings in a thesis/report/book. Examples of approachable books include: “Engineering Culture: Control and Commitment in a High-Tech Corporation” by Kunda, who studied a large high-tech company in the mid-1980s; “No-Collar: The Humane Workplace and its Hidden Costs” by Ross, who studied an internet startup that had just IPO’ed, and “Coding Freedom: The Ethics and Aesthetics of Hacking” by Coleman, who studied hacker culture.

Linguistic anthropology is the field whose researchers are mostly likely to match developers’ preconceived ideas about what humanities academics talk about. If I had been educated in an environment where Greek and nineteenth century philosophers were the reference points for any discussion, then I too would use this existing skill set in my discussions of source code (philosophers of source code did not appear until the twentieth century). Who wouldn’t want to apply hermeneutics to the interpretation of source code (the field is known as Critical code studies)?

It does not help that the software knowledge of many of the academics appears to have been acquired by reading computer books from the 1940s and 1950s.

The most approachable linguistic anthropology book I have found, for developers, is: The Philosophy of Software Code and Mediation in the Digital Age by Berry (not that I have skimmed many).

Letter to Anneliese Dodds on the invasion of Ukraine by Russia

Tim Pizey from Tim Pizey

Dear Anneliese Dodds,

I learn from the BBC (https://www.bbc.co.uk/news/58888451) that "The UK is to phase out Russian oil by the end of the year" and "Russian imports account for 8% of total UK oil demand".

8% is a small amount and the end of the year is a long time in the future. We need immediate action to change Russia's course. Please use all your influence to this end.

Some suggestions, as a minimum:

  • Stop all petrochemical purchases from Russia, and requiring this of multinationals
  • Expulsion of all remaining Russian banks from SWIFT
  • Make it unlawful to insure a Russian enterprise
  • Seizure and forfeiture of all Russian assets within the UK and its dominions
  • Motion to remove Russia from the UN Security Council

There are many more things which could and should be done, by January 2023 there will be no Ukraine to defend.

Yours sincerely,
Tim Pizey

Galactic North (a review)

Paul Grenyer from Paul Grenyer

Galactic North

Alastair Reynolds
ISBN-13: ‎ 978-0575083127

Galactic North is a group of short stories set in the Revelation Space universe starting at it’s very beginning and stretching right to it’s end.


Great Wall of Mars


I reread the Great Wall of Mars after the Inhibitor Phase to remind me of some of Warren Clavian’s back story. It didn’t disappoint. I should have read Great Wall of Mars again before Inhibitor Phase, but hindsight is a wonderful thing. At least I’ve remembered why Nevile hated his brother and how he was betrayed by him, why Nevile defected to the conjoiners and how Felka fits in. One small story explains so much of why things happened in several of the other stories including Absolution Gap.


Glacial
 

Glacial adds little to the overall story, but does help to explain how the relationship between Clavian, Galiana and Felka developers and how it becomes so strong. Glacial is really an opportunity for Alastair Reynolds to explore the concept of a thinking, possible sentient planet.  He does this, as always, by hinting throughout at the bigger picture and keeping you reading.


A Spy in Europa

 
Not sure what I think of this one. Seemed a bit pointless. Not very nice characters who all stabbed each other in the back. Some interesting science tho and provided a backdrop and context for Grafenwalder's Bestiary.


Weather

 
What a fantastic standalone story this is with some great characters who demonstrate that not all Ultras are cut-throat. There’s lots more detail about conjoiners here and the secret of how C-drives are managed is revealed, but that’s not the darkest secret.


Dilation Sleep

 
I was disappointed in this story until I read the notes at the end and realised it was the first story written in the revelation space universe and that it introduced some key aspects, such as Chasm City. It doesn’t really add anything to the overall story, but has some interesting insights into refersleep.
 

Grafenwalder's Bestiary

 
Some of the best stories are those which are difficult to read due to the behaviour of some of the characters. When they do things you can’t understand the motivation for and could not imagine doing yourself. In this story it’s cruelty, deception and revenge and I loved it.


Nightingale

I do wonder how Alastair Reynolds thinks up these horrors, but they are glorious. This story is particularly horrible at the end. The evil computer was far worse than anything in the Resident Evil series, with undertones of Hal 9000. There’s exploration in the story, battle, weapons and the sort of intrigue which makes it difficult to put down. 


Galactic North 

This should be expanded to a novel, or at least a novella. There’s scope for so much evolution, especially with the greenfly and how they come to take over. I couldn’t put this down, and wouldn’t have done it if my Kindle hadn’t died a few pages before the end!

In some ways it’s a shame that Alastair Reynolds has put a hard limit on the timeline of Revelation Space, but I loved it! I reread Galactic North to understand the comments at the end of Inhibitor Phase and the Nest Builders. I should have read it first. And I should have a Revelation Space timeline on my wall.


Study of developers for the cost of a phase I clinical drug trial

Derek Jones from The Shape of Code

For many years now, I have been telling people that software researchers need to be more ambitious and apply for multi-million pound/dollar grants to run experiments in software engineering. After all, NASA spends a billion or so sending a probe to take some snaps of a planet and astronomers lobby for $100million funding for a new telescope.

What kind of experimental study might be run for a few million pounds (e.g., the cost of a Phase I clinical drug trial)?

Let’s say that each experiment involves a team of professional developers implementing a software system; call this a Project. We want the Project to be long enough to be realistic, say a week.

Different people exhibit different performance characteristics, and the experimental technique used to handle this is to have multiple teams independently implement the same software system. How many teams are needed? Fifteen ought to be enough, but more is better.

Different software systems contain different components that make implementation easier/harder for those involved. To remove single system bias, a variety of software systems need to be used as Projects. Fifteen distinct Projects would be great, but perhaps we can get away with five.

How many developers are on a team? Agile task estimation data shows that most teams are small, i.e., mostly single person, with two and three people teams making up almost all the rest.

If we have five teams of one person, five of two people, and five of three people, then there are 15 teams and 30 people.

How many people will be needed over all Projects?

15 teams (30 people) each implementing one Project
 5 Projects, which will require 5*30=150 people (5*15=75 teams)

How many person days are likely to be needed?

If a 3-person team takes a week (5 days), a 2-person team will take perhaps 7-8 days. A 1-person team might take 9-10 days.

The 15 teams will consume 5*3*5+5*2*7+5*1*9=190 person days
The  5 Projects will consume              5*190=950 person days

How much is this likely to cost?

The current average daily rate for a contractor in the UK is around £500, giving an expected cost of 190*500=£475,000 to hire the experimental subjects. Venue hire is around £40K (we want members of each team to be co-located).

The above analysis involves subjects implementing one Project. If, say, each subject implements two, three or four Projects, one after the other, the cost is around £2million, i.e., the cost of a Phase I clinical drug trial.

What might we learn from having subjects implement multiple Projects?

Team performance depends on the knowledge and skill of its members, and their ability to work together. Data from these experiments would be the first of their kind, and would provide realistic guidance on performance factors such as: impact of team size; impact of practice; impact of prior experience working together; impact of existing Project experience. The multiple implementations of the same Project created provide a foundation for measuring expected reliability and theories of N-version programming.

A team of 1 developer will take longer to implement a Project than a team of 2, who will take longer than a team of 3.

If 20 working days is taken as the ballpark period over which a group of subjects are hired (i.e., a month), there are six team size sequences that one subject could work (A to F below); where individual elapsed time is close to 20 days (team size 1 is 10 days elapsed, team size 2 is 7.5 days, team size 3 is 5 days).

Team size    A      B      C      D      E      F
    1      twice   once   once  
    2                     once  thrice  once
    3             twice                twice   four

The cost of hiring subjects+venue+equipment+support for such a study is likely to be at least £1,900,000.

If the cost of beta testing, venue hire and research assistants (needed during experimental runs) is included, the cost is close to £2.75 million.

Might it be cheaper and simpler to hire, say, 20-30 staff from a medium size development company? I chose a medium-sized company because we would be able to exert some influence over developer selection and keeping the same developers involved. The profit from 20-30 people for a month is not enough to create much influence within a large company, and a small company would not want to dedicate a large percentage of its staff for a solid month.

Beta testing is needed to validate both the specifications for each Project and that it is possible to schedule individuals to work in a sequence of teams over a month (individual variations in performance create a scheduling nightmare).

On A Generally Fractal Family – student

student from thus spake a.k.

Recently, my fellow students and I have been caught up in the craze that is sweeping through the users of Professor B------'s clockwork calculating engine; namely the charting of sets of two dimensional points that have fractal planar boundaries, being those that in some sense have a fractional dimension. Of particular interest have been the results of repeated applications of quadratic functions to complex numbers; specifically in measuring how quickly, if at all, they escape a region surrounding the starting point, by which charts may be constructed that many of the collegiate consider so delightful as to constitute art painted by mathematics itself!

User Stories by Example tutorials now Free

Allan Kelly from Allan Kelly

My User Stories by Example tutorial series is now free.

There are five tutorials which include video lectures, worked examples with on real user stories and exercises covering user stories basics, acceptance criteria, story splitting, story refactoring and more.

The series is based on my Little Book of Requirements and User Stories – the audio files for the book are there too but you will have to pay for them. If you are an Audible subscriber you can get the book there as part of you subscription. Print book, eBook and audio book are all available at Amazon.


Subscribe and download Continuous Digital for free

The post User Stories by Example tutorials now Free appeared first on Allan Kelly.

Growth in FLOPS used to train ML models

Derek Jones from The Shape of Code

AI (a.k.a. machine learning) is a compute intensive activity, with the performance of trained models being dependent on the quantity of compute used to train the model.

Given the ongoing history of continually increasing compute power, what is the maximum compute power that might be available to train ML models in the coming years?

How might the compute resources used to train an ML model be measured?
One obvious answer is to specify the computers used and the numbers of days used they were occupied training the model. The problem with this approach is that the differences between the computers used can be substantial. How is compute power measured in other domains?

Supercomputers are ranked using FLOPS (floating-point operations per second), or GigFLOPS or PetaFLOPS (10^{15}). The Top500 list gives values for R_{max} (based on benchmark performance, i.e., LINNPACK) and R_{peak} (what the hardware is theoretically capable of, which is sometimes more than twice R_{max}).

A ballpark approach to measuring the FLOPS consumed by an application is to estimate the FLOPS consumed by the computers involved and multiply by the number of seconds each computer was involved in training. The huge assumption made with this calculation is that the application actually consumes all the FLOPS that the hardware is capable of supplying. In some cases this appears to be the metric used to estimate the compute resources used to train an ML model. Some published papers just list a FLOPS value, while others list the number of GPUs used (e.g., 2,128).

A few papers attempt a more refined approach. For instance, the paper describing the GPT-3 models derives its FLOPS values from quantities such as the number of parameters in each model and number of training tokens used. Presumably, the research group built a calibration model that provided the information needed to estimate FLOPS in this way.

How does one get to be able to use PetaFLOPS of compute to train a model (training the GPT-3 175B model consumed 3,640 PetaFLOP days, or around a few days on a top 8 supercomputer)?

Pay what it costs. Money buys cloud compute or bespoke supercomputers (which are more cost-effective for large scale tasks, if you have around £100million to spend plus £10million or so for the annual electricity bill). While the amount paid to train a model might have lots of practical value (e.g., can I afford to train such a model), researchers might not be keen to let everybody know how much they spent. For instance, if a research team have a deal with a major cloud provider to soak up any unused capacity, those involved probably have no interest in calculating compute cost.

How has the compute power used to train ML models increased over time? A recent paper includes data on the training of 493 models, of which 129 include estimated FLOPS, and 106 contain date and model parameter data. The data comes from published papers, and there are many thousands of papers that train ML models. The authors used various notability criteria to select papers, and my take on the selection is that it represents the high-end of compute resources used over time (which is what I’m interested in). While they did a great job of extracting data, there is no real analysis (apart from fitting equations).

The plot below shows the FLOPS training budget used/claimed/estimated for ML models described in papers published on given dates; lines are fitted regression models, and the colors are explained below (code+data):

FLOPS consumed training ML models over time.

My interpretation of the data is based on the economics of accessing compute resources. I see three periods of development:

  1. do-it yourself (18 data points): During this period most model builders only had access to a university computer, desktop machines, or a compute cluster they had self-built,
  2. cloud (74 data points): Huge on demand compute resources are now just a credit card away. Researchers no longer have to wait for congested university computers to become available, or build their own systems.

    AWS launched in 2006, and the above plot shows a distinct increase in compute resources around 2008.

  3. bespoke (14 data points): if the ML training budget is large enough, it becomes cost-effective to build a bespoke system, e.g., a supercomputer. As well as being more cost-effective, a bespoke system can also be specifically designed to handle the characteristics of the kinds of applications run.

    How might models trained using a bespoke system be distinguished from those trained using cloud compute? The plot below shows the number of parameters in each trained model, over time, and there is a distinct gap between 10^{10} and 10^{11} parameters, which I assume is the result of bespoke systems having the memory capacity to handle more parameters (code+data):

    Number of parameters in ML models over time.

The rise in FLOPS growth rate during the Cloud period comes from several sources: 1) the exponential decline in the prices charged by providers delivers researchers an exponentially increasing compute for the same price, 2) researchers obtaining larger grants to work on what is considered to be an important topic, 3) researchers doing deals with providers to make use of excess capacity.

The rate of growth of Cloud usage is capped by the cost of building a bespoke system. The future growth of Cloud training FLOPS will be constrained by the rate at which the prices charged for a FLOP decreases (grants are unlikely to continually increase substantially).

The rate of growth of the Top500 list is probably a good indicator of the rate of growth of bespoke system performance (and this does appear to be slowing down). Perhaps specialist ML training chips will provide performance that exceeds that of the GPU chips currently being used.

The maximum compute that can be used by an application is set by the reliability of the hardware and the percentage of resources used to recover from hard errors that occur during a calculation. Supercomputer users have been facing the possibility of hitting the wall of maximum compute for over a decade. ML training is still a minnow in the supercomputer world, where calculations run for months, rather than a few days.

Migrating source code from RCS to Mercurial

Timo Geusch from The Lone C++ Coder's Blog

Version control system migrations are a fact of life for developers in any longer lived codebase. In fact, I’ve had a hand in quite a few migrations as newer, more workable version control systems became available. Also, like a lot of developers, I’ve got fragments of source code dating back quite some years floating around on various servers and development machines of mine. Not necessarily code that is still being used, but still code that I don’t want to just delete forever.

The Agile Elephant and the agile mindset

Allan Kelly from Allan Kelly

African Elephant

Confession: I’ve been avoiding the words “agile mindset” for some time because I don’t know what it is. And, completely by coincidence, I’ve recently had a couple of encounters that have caused me to think again. So let me explain…

I repeatedly find myself wrestling with the question “What is agile?” The question came up recently in a new form when I was invited to give a talk on “The Agile Mindset.” I appealed for help on LinkedIn. I got some great answers and the diversity of answers confirmed what I though: it is hard to describe “the agile mindset” in a short or generally agreed form.

The first problem is that to explain “the Agile mindset” one first has to agree what agile is, and is not. I have my own view but I know there is a diversity of opinion so I find it useful to describe “Agile” with the story of the blind men examining an elephant: one feels the leg and says “This is a mighty tree”, another feels the tusk and says “It is a strong sword”, another the trunk and says “It is a strong snake” and so on. Each interprets the part they encounter as the whole yet the whole, to one who has never seen an elephant, can be hard to comprehend.

Illustration from the Natural History Museum, London

The same is true for agile.

The literalist looks in the dictionary and says “Agile is about being fast, reactive and responding to the outside”, the engineer looks at agile and says “It is about doing quality work so we may deliver more”, the Scum aficionado says “It is about high performing teams and alignment”, the Lean thinker says “It is about reducing work in progress and simplifying workflows” and the management consultant says “It is about delivering more with less.”

All are right, none is wrong. And while that is a problem in describing what agile is it is also a strength. Agile is multi-faceted and offers “something for everyone.” While different people emphasis different things it also means the whole is more than the sum of the parts. If you can harness high performing teams, with engineering quality, low WIP and reactive processes then you can deliver the fabled faster, better, cheaper.

But that also makes it hard.

It also goes some way to explaining why “Agile Coaches” never agree: each has their own interpretation of how to put those pieces together to make the whole – to change metaphor, everyone approaches the jigsaw differently.

And again that is right because every jigsaw, every application of agile, exists in a unique context and must be faced on its own terms – to quote Tolstoy: “All happy families are the same, all unhappy families are unhappy in their own unique way.” (And long time readers might notice I just contradicted myself.)

And one important reason why the jigsaw is always different is: in completing the last jigsaw, and since completing it, you, and everyone else as learned, the bodies may be the same but the people – and their minds – are different.

Ultimately, I still claim “Agile” is learning, specifically organizational learning: the thesis I laid out in my first book over 10 years ago Changing Software Development.

Hence I say: The only thing you can do wrong in agile is work the same as you did three months ago. To be agile one should always be learning and changing as a result of that learning.

I should explain that some more in another post, and I’ll have more to say about the agile mindset soon.


Subscribe to my blog newsletter and download Continuous Digital for free

The post The Agile Elephant and the agile mindset appeared first on Allan Kelly.