Bulldozer00's Blog

The anecdotal evidence is overwhelming. Agile methods can work really well for many small teams and small projects. However, no matter what the expert, high-profile, “coaches” purport, the jury is still out regarding its scalability to large teams and large projects. In “How even agile development couldn’t keep this mega-project on track“, Nick Heath showcases the British disaster known as the £2.4bn “Universal Credit Programme“.

First, the sad fact:

…the UK government has had to write off at least £34m on the programme and delay the national launch for the project. The department in charge of the project, the Department for Work and Pensions (DWP), can’t guarantee the remainder of the £303m it has spent on the project so far will offer “good value” it said.

From the rest of Nick’s story, it becomes clear that agile methods weren’t really used to develop the software:

There was a two-year gap between the DWP starting the project design and build process, and the system going live in 2013.

The DWP experienced problems incorporating the agile approach into existing contracts, governance and assurance structures.

That second point is key. No matter how much a big org wants to be “agile“, it is heavily constrained by the hierarchical structures, stature-obsessed mindsets, byzantine processes, and form-filled procedures entrenched within not only itself, but also within its suppliers and customers. It’s a classic “system” problem where futzing around with one component may crash the whole system because of hardened interfaces and skin tight coupling.

As the figure below shows, attempting to “agilize” a large component within an even larger, waterfall-centric, system creates impedance mismatches at every interface. The greater the mismatch, the less productive the system becomes. Information flow and understanding between components bog down while noise and distortion overwhelm the communication channels. In the worse case, the system stops producing value-added output and it would have been better to leave the old, inefficient, waterfall-centric system intact.

The only chance an agile-wanna-be component has at decoupling itself from the external waterfall insanity is to covertly setup a two-faced, agile<->waterfall protocol converter for each of its external interfaces. Good luck pullin’ that stunt off.