Archive
Variable Sized MWs
Rewritten in “old school” terminology, the five Scrum process events can be expressed as follows:
- Sprint Planning = Requirements definition and capture
- Sprint = Requirements analysis, design, coding, unit testing, integration testing, code review
- Daily Stand Up = Daily status meeting
- Sprint Review = Post-mortem
- Sprint Retrospective = Continuous process improvement
So, someone with an intentionally warped mind like BD00 may interpret a series of Scrum sprints as nothing more than a series of camouflaged Mini-Waterfalls (MW).
But ya know what? Executing a project as a series of MWs may be a good thing – as long as an arbitrary, fixed-size, time-box is not imposed on the team. After all, since everything else is allowed to dynamically change during a Scrum project, why not the size of the Sprint too?
Instead of estimating what features can be done in the next 30 days, why not simply estimate how many days will be needed to complete the set of features that marks the transition into the next MW? If, during the MW, it is learned that the goal won’t be achieved, then in addition to cancelling the MW outright, two other options can be contemplated:
- Extend the length of the MW
- Postpone the completion of one or more of the features currently being worked on
Understood, Manageable, And Known.
Our sophistication continuously puts us ahead of ourselves, creating things we are less and less capable of understanding – Nassim Taleb
It’s like clockwork. At some time downstream, just about every major weapons system development program runs into cost, schedule, and/or technical performance problems – often all three at once (D’oh!).
Despite what their champions espouse, agile and/or level 3+ CMMI-compliant processes are no match for these beasts. We simply don’t have the know how (yet?) to build them efficiently. The scope and complexity of these Leviathans overwhelms the puny methods used to build them. Pithy agile tenets like “no specialists“, “co-located team“, “no titles – we’re all developers” are simply non-applicable in huge, multi-org programs with hundreds of players.
Being a student of big, distributed system development, I try to learn as much about the subject as I can from books, articles, news reports, and personal experience. Thanks to Twitter mate @Riczwest, the most recent troubled weapons system program that Ive discovered is the F-35 stealth fighter jet. On one side, an independent, outside-of-the-system, evaluator concludes:
The latest report by the Pentagon’s chief weapons tester, Michael Gilmore, provides a detailed critique of the F-35’s technical challenges, and focuses heavily on what it calls the “unacceptable” performance of the plane’s software… the aircraft is proving less reliable and harder to maintain than expected, and remains vulnerable to propellant fires sparked by missile strikes.
On the other side of the fence, we have the $392 billion program’s funding steward (the Air Force) and contractor (Lockheed Martin) performing damage control via the classic “we’ve got it under control” spiel:
Of course, we recognize risks still exist in the program, but they are understood and manageable. – Air Force Lieutenant General Chris Bogdan, the Pentagon’s F-35 program chief
The challenges identified are known items and the normal discoveries found in a test program of this size and complexity. – Lockheed spokesman Michael Rein
All of the risks and challenges are understood, manageable, known? WTF! Well, at least Mr. Rein got the “normal” part right.
In spite of all the drama that takes place throughout a large system development program, many (most?) of these big ticket systems do eventually get deployed and they end up serving their users well. It simply takes way more sweat, time, and money than originally estimated to get it done.
Long Live The King!
Even though it has a title and cover design that only a Harvard MBA could love, I picked up Robert Austin’s “Measuring and Managing Performance in Organizations” on a twitter tip from Torbjörn Gyllebring. As soon as I cracked the cover, I knew it was gonna be a classic. The foreword was written by one of my all time favorite software authors, Tom DeMarco.
Mr. Austin discovered perhaps the first recorded instance of the well worn “schedule is king!” management law:
Such scenarios, in which program managers or contractors attend to measurements of timeliness of delivery to the exclusion of all else, are reported as early as 1882. In that year, the newly built U.S.S. Omaha was discovered to have onboard-coal-room for only four days’ steaming; in the rush to stay on schedule, no one had been willing to force notice of this defect at a high enough level to ensure its correction.
Since there seems to be no worthy candidate on the horizon capable of dethroning the king, expect this monarch to live long indeed.
Every once in a blue moon, I finish a book so engrossing that I immediately reread it before cracking open a different one. Mr. Austin’s MAMPIO is one of those gems and I’m well into my second romp through it. Since it’s loaded with a gazillion ideas for blog posts, expect more over-the-top BD00 distortions to come. W00t!
Impedance Mismatch
The anecdotal evidence is overwhelming. Agile methods can work really well for many small teams and small projects. However, no matter what the expert, high-profile, “coaches” purport, the jury is still out regarding its scalability to large teams and large projects. In “How even agile development couldn’t keep this mega-project on track“, Nick Heath showcases the British disaster known as the £2.4bn “Universal Credit Programme“.
First, the sad fact:
…the UK government has had to write off at least £34m on the programme and delay the national launch for the project. The department in charge of the project, the Department for Work and Pensions (DWP), can’t guarantee the remainder of the £303m it has spent on the project so far will offer “good value” it said.
From the rest of Nick’s story, it becomes clear that agile methods weren’t really used to develop the software:
There was a two-year gap between the DWP starting the project design and build process, and the system going live in 2013.
The DWP experienced problems incorporating the agile approach into existing contracts, governance and assurance structures.
That second point is key. No matter how much a big org wants to be “agile“, it is heavily constrained by the hierarchical structures, stature-obsessed mindsets, byzantine processes, and form-filled procedures entrenched within not only itself, but also within its suppliers and customers. It’s a classic “system” problem where futzing around with one component may crash the whole system because of hardened interfaces and skin tight coupling.
As the figure below shows, attempting to “agilize” a large component within an even larger, waterfall-centric, system creates impedance mismatches at every interface. The greater the mismatch, the less productive the system becomes. Information flow and understanding between components bog down while noise and distortion overwhelm the communication channels. In the worse case, the system stops producing value-added output and it would have been better to leave the old, inefficient, waterfall-centric system intact.
The only chance an agile-wanna-be component has at decoupling itself from the external waterfall insanity is to covertly setup a two-faced, agile<->waterfall protocol converter for each of its external interfaces. Good luck pullin’ that stunt off.
All Forked Up
I dunno who said it, but paraphrasing whoever did:
Science progresses as a succession of funerals.
Even though more accurate and realistic models that characterize the behavior of mass and energy are continuously being discovered, the only way the older physics models die out is when their adherents kick the bucket.
The same dictum holds true for software development methodologies. In the beginning, there was the Traditional (a.k.a waterfall) methodology and its formally codified variations (RUP, MIL-STD-498, CMMI-DEV, your org’s process, etc). Next, came the Agile fork as a revolutionary backlash against the inhumanity inherent to the traditional way of doing things.
The most recent fork in the methodology march is the cerebral SEMAT (Software Engineering Method And Theory) movement. SEMAT can be interpreted (perhaps wrongly) as a counter-revolution against the success of Agile by scorned, closet traditionalists looking to regain power from the agilistas.
On the other hand, perhaps the Agile and SEMAT camps will form an alliance and put the final nail in the coffin of the old traditional way of doing things before its adherents kick the bucket.
SEMAT co-creator Ivar Jacobson seems to think that hitching SEMAT to the Agile gravy train holds promise for better and faster software development techniques.
Who knows what the future holds? Is another, or should I say, “the next“, fork in the offing?
The Drooping Progress Syndrome
When a new product development project kicks off, nobody knows squat and there’s a lot of fumbling going on before real progress starts to accrue. As the hardware and software environment is stitched into place and initial requirements/designs get fleshed out, productivity slowly but surely rises. At some point, productivity (“velocity” in agile-ese) hits a maximum and then flattens into a zero slope, team-specific, cadence for the duration. Thus, one could be led to believe that a generic team productivity/progress curve would look something like this:
In “The Year Without Pants“, Scott Berkun destroys this illusion by articulating an astute, experiential, observation:
This means that at the end of any project, you’re left with a pile of things no one wants to do and are the hardest to do (or, worse, no one is quite sure how to do them). It should never be a surprise that progress seems to slow as the finish line approaches, even if everyone is working just as hard as they were before. – Scott Berkun
Scott may have forgotten one class of thing that BD00 has experienced over his long and un-illustrious career – things that need to get done but aren’t even in the work backlog when deployment time rolls in. You know, those tasks that suddenly “pop up” out of nowhere (BD00 inappropriately calls them “WTF!” tasks).
Nevertheless, a more realistic productivity curve most likely looks like this:
If you’re continuously flummoxed by delayed deployments, then you may have just discovered why.
Related articles
- The Year Without Pants: An interview with author Scott Berkun (oldienewbies.wordpress.com)
- Scott Berkun Shares Advice for Writers Working Remotely (mediabistro.com)
- A Book in 5 Minutes: “The Year without Pants: WordPress.com and the Future of Work” (tech.co)
A Concrete Agile Practices List
Finally, I found out what someone actually thinks “agile practices” are. In “What are the Most Important and Adoption-Ready Agile Practices?”, Shane-Hastie presents his list:
Kudos to Shane for putting his list out there.
Ya gotta love all the “explicit definition of done” entries (“Aren’t you freakin’ done yet?“). And WTF is “Up front architecture” doing on the list? Isn’t that a no-no in agile-land? Shouldn’t it be “emergent architecture“? And no kanban board entry? What about burn down charts?
Alas, I can’t bulldozify Shane’s list too much. After all, I haven’t exposed my own agile practices list for scrutiny. If I get the itch, maybe I’ll do so. What’s on your list?
Scrummin’ For The Ilities
Whoo hoo! The product owner has funded our project and we’ve spring-loaded our backlog with user stories. We’re off struttin’ and scrummin’ our way to success:
But wait! Where are the “ilities” and the other non-functional product requirements? They’re not in your backlog?
Uh, we don’t do “ilities“. It’s not agile, it’s BDUF (Big Design Up Front). We no need no stinkin’ arkitects cuz we’re all arkitects. Don’t worry. These “ilities” thingies will gloriously emerge from the future as we implement our user stories and deliver working increments every two weeks – until they stop working.
Big, No, And Just Enough
Assume (which is a bad thing to do) that you can characterize a project by three activities: Planning, Designing, and Building. Next, assume (which is still a bad thing to do) that the planning and designing activities can each be sub-characterized by “Big“, “No“, and “Just Enough“.
Forgetting about what “Just Enough” exactly means, check out this methodology-agnostic probability density function:
Do you think the curve is right? You better not. The sample size BD00 used to generate it is zero,
One Or Many
The figure below models the increase in team stress level versus time for waterfall and time-boxed projects. As a project nears a delivery date, the stress levels increase dramatically as the team fixes turds, integrates individually developed features into the whole, and takes care of the boring but important stuff that nobody wanted to do earlier.
One tradeoff between the two types of projects is maximum stress level vs. number of stressful events. The maximum level of experienced stress is much higher for waterfall than any one time-boxed sprint, but it only occurs once as opposed to monthly. Pick your poison: a quick death by guillotine or a slow death by a thousand cuts.
Agilistas would have everyone believe that time-boxed projects impose a constant but very low level of healthy stress on team members while waterfall quagmires impose heart-attack levels of stress on the team. They may be right, because…..
(In case you haven’t noticed, BD00 is feeling the need to use the Einstein pic above more and more in his posts. I wonder why that is.)
Why am I here?
WTF?
Meh!
D'oh!
My BTC Address
Bulldozer00's Blog 