Frances Buontempo from BuontempoConsulting
A write up of my notes: they may or may not make any sense.Keynote: Jez Humble "What I Learned From Three Years Of Sciencing The Cr*p Out Of Continuous Delivery" or "All about SCIENCE"
Suverys
Surveys are measures looking for latent constructs for feelings and similar - see psychometrics.Surveys need a hypothesis to test and should be worded carefully.
Consider discriminant and convergent validity.
Test for false positives.
Consider the Westrum toypology.
With 6 axes (rows) scaled across three columns: pathological, bureaucratic, generative you can start spotting connections.
Pathological | Bureaucratic | Generative |
Power Oriented | Rule Oriented | Performance Oriented |
Low cooperation | Modest cooperation | High cooperation |
Messengers shot | Messengers neglected | Messengers trained |
Responsibilities shirked | Narrow responsibilities | Risks are shared |
Bridging discouraged | Bridging tolerated | Bridging encouraged |
Failure leads to scapegoating | Failure leads to justice | Failure leads to inquiry |
Novelty crushed | Novelty leads to problems | Novelty implemented |
For example "Failure leads to" has three different options: scapegoating, justice or inquiry. Where does your org come out for each question? If they say "It's all Matt's fault" and sack Matt that won't avoid mistakes happening again. Blameless postmortems are important.
In general for surveys, use a Likert type scale - use clearly worded statements on a scale, allowing numerical analysis. See if your questions "load together" (or bucket). Maybe spotting what's gone wrong with some software buckets into notification from outside (customers etc) and notification from inside (alerts etc).
Consider CMV, CMB - common method variance or bias. Look for early versus late respondents.
In fact take this year's https://puppetlabs.com/blog/2016-state-devops-survey-here
IT performance
Does your company have a culture of "autonomy, mastery, purpose"? What motivates us? [See Pink]
How do we measure IT performance? Consider lead time, release frequency, time to restore, change failure rate...
Going faster doesn't mean you break things, it actually makes you *more* stable, if you look at the data [citation needed]
"Bi-modal IT" is wrong: watch out for Jez's upcoming blog about "fast doesn't compromise safety"
Do we still want to work in the dark-ages of manual config and no test automation?
We claim we are doing continuous integration (CI) by redefining CI. Do devs merge to trunk daily? Do you have tests? Do you fix the build if it goes red?
Aside: "Surveys are a powerful source of confirmation bias"
Question: Can we work together when things go wrong?
Do you have peer reviewed changes? (Mind you, change advisory boards)
Science again (well, stats)
SEM: structured equation modelling: use this to avoid spurious correlations.Apparently 25% of people do TDD - it's the lost XP practice. TDD forces you to write code in testable ways: it's not about the tests.
How good are your tests? Consider mutation testing e.g. Ivan Moore's Jester
Change advisory boards don't work. They obviously impact throughput but have negligible impact on stability. Jez suggested the phrase "Risk management theatre".
Ian Watson and Chris Covell "Steps closer to awesome"
They work at Call Credit (used to be part of the Skipton building soc) and talked about how to change an organisation.Their hypothesis: "You already have the people you need."
"Metal as a service" sneaked a mention, since some people were playing buzz-word bingo.
Question: what would make this org "nirvana"?
They started broadcasting good (and bad) things to change the culture. e.g. moving away from a fear of failure. Having shared objectives helped.
We are people, not resources. "Matrix management" (queue obvious slides) - not a good thing. Be the "A" team instead. (Or the goonies).
The environment matters. They suggested blowing up a red balloon each time you are interrupted for 15 seconds or more, giving a visual aid of the distractions.
They mentioned "Death to manual deployments" being worth reading.
They said devs should never have access to prod.
You need centres of excellence: peer pressure helps.
They have new bottlenecks: "two speed IT" .... the security team should be enablers not the police.
They mentioned the "improvement kata"
They said you need your ducks in a straight line == a backlog of good stories.
Gary Frost "Financial Institutions Carry Too Much Risk, It’s Time To Embrace Continuous Delivery"
It brought about a segregation of duties and lots of change control review. "runbooks" This is still high risk. There have been lots of breeches from IT departments e.g. Knight Capital, NatWest (3 times).
Why are we still failing, despite these "safety measures"?
What are the blockers? Silos. Move to collaborative environments.
Gustavo Elias "How To Deal With A Hot Potato"
He was landed with legacy code that was deeply flawed, had multiple responsibilities and high maintenance costs. In fact he calculated these costs and told management, For example, with downtime for deployment and 40 minutes to restarted calculate the cost at over £500 per day per dev.- Re-architect
- Reach zero downtime
- Detach from the old release cycle
In the end, be proud.
Pete Marshall "Achieving Continuous Delivery In A Legacy Environment"
They had DNS load balancing, "interesting stand-ups" (nobody cared), no monitoring.
He changed nant to msbuild.
He used the strangle pattern.
Sally Goble "What do you do if you don't do testing?"
From QA at The GuardianThey previously has a two-week release cycle, with a staging environment and lots of manual testing.
Steve Elliott "Measure everything, not just production"
He pointed us at github