Archive
How Can We Make This Simpler?
The biggest threat to success in software development is the Escalation of Unessential Complexity (EUC). Like stress is to humans, EUC is a silent project/product killer lurking in the shadows – ready to pounce. If your team isn’t constantly asking “how can we make this simpler?” as time ticks by and money gets burned, then your project and product will probably get hosed somewhere downstream.
But wait! It’s even worse than you think. Asking “how can we make this simpler?” applies to all levels and areas of a project: requirements, architecture, design, code, tools, and process. Simply applying continuous inquiry to one or two areas might delay armageddon, but you’ll still probably get hosed.
If you dwell in a culture that reveres or is ignorant of EUC, then you most likely won’t ever hear the question “how can we make this simpler?” getting asked.
But no need to fret. All ya gotta do to calm your mind is embrace this Gallism :
In complex systems, malfunction and even total malfunction may not be detectable for long periods, if ever – John Gall
The New G-Spot
Before reading any further, please contemplate this Box cutter:
Essentially, all models are wrong, but some are useful- George Box
Now, let’s start with a pair of “model” problem-system and control-system templates:
Next, let’s hypothesize a new composite system that you designed, built, and deployed to “solve a vexing socio-technical problem“. You diligently employed the dorky BD00 templates, your brain, and your positional power to form a “semi–effective” marriage between your specific problem (indicated by the blue, scoped boundary) and your specific controller (indicated by non-white components):
Although your new system may appear to be alleviating (or least containing) the problem, looks can be (and usually are) deceiving. Before you pat yourself on the back for being a “problem solver“, please contemplate these Gall-isms:
New systems generate new problems – John Gall
Systems Develop Goals Of Their Own The Instant They Come Into Being – John Gall
Intra-system Goals Come First – John Gall
Thus, according to the Gallster, the a-priori and noble “G” you designed into your fabulous controller subsystem will eventually be usurped (sooner rather than later) by the emergent “G” of the new composite system entity. Sometimes the new “G” totally obscures the original “G” so thoroughly that, given enough time, nobody can even remember what the original “G” was. D’oh!
And what might this new system “G-Spot” be? Could it be to perpetuate and maybe even exacerbate the original problem so that the new system (especially the controller subsystem) can grow and thrive? After all, if the problem gets solved, then there will be no more need for the controller subsystem. In effect, the new system would be devouring itself as it solved the problem. But wait! That can’t happen because it would be a fruitless attempt to violate this Gall-Shirky pair:
Systems Tend To Grow, And As They Grow, They Encroach – John Gall
Institutions Tend To Preserve The Problem To Which They Are The Solution – Clay Shirky
An alternative BD00 quote rip-off is:
Systems, like people, tend not to consume themselves as food.
Can you concoct another alternative rip-off quote?
Glad To Be Of Service
Much of my thinking on hierarchy and unconsciously veiled corpo-insanity is founded on the ideas of systems thinkers and cyberneticians like Ackoff, Deming, Beer, Ashby, Wiener, Forrester, Meadows, Senge, Wheatley, Warfield, Bateson, Gall, Powers, etc. But mostly, my dirty thinking is rooted in the life work of William T. Livingston and his most influential mentor, Rudy Starkermann.
Over the years, Bill has always claimed that his work on socio-technical dysfunction may not be right, but it is irrefutable because it is derived from natures laws (mostly thermodynamics and control theory). And in walking his talk, Bill constantly solicits feedback and asks for counterexamples that disprove his theories.
After I discovered and wrote about Valve Inc, I threw this skunk on my friend’s table:
Here’s Bill’s response and my response to his response:
With his approval, which I have no doubt whatsoever that I’ll receive, I’ll try to decode and post the results of Bill’s research when I get it.
Related articles
- D4P Has Been Hatched (bulldozer00.com) ( Download the D4P book for free)
- D4P And D4F (bulldozer00.com)
- D4P4D (bulldozer00.com)
- D4P4D Tweetfest (bulldozer00.com)
Come To Papa!
I recently listened to a fascinating podcast interview of Valve Inc‘s “economist-in-residence“, Yanis Varoufakis. According to Yanis, the company is still organizationally flat after 17 years of existence.
The thought early on at Valve was that the maximum limit to flatness would be around 50-60 people. Above that, in order to keep the wheels from falling off, some form of hierarchy would be required for concerted coordination. However, currently at 300+ employees, Valve has managed to blow through that artificial barrier and remain flat. Mind you, this is not a company solely made up of like-thinking engineers. There are also artists, animators, writers, and accountants running around like a herd of cats inefficiently doing the shit that brings in $1B in revenue each year.
According to Yanis, in order to maintain their egalitarian culture, Valve can’t afford to grow too quickly. That’s because they have to deprogram people who are hired in from hierarchical borgs as former bosses who expect others to work for them, and former workers who expect to be “directed” by a boss. If Valve didn’t do this, their culture would get eaten alive by the pervasive and mighty command-and-control mindset. The spontaneity, creativity, and togetherness that power their revenue machine would be lost forever.
Nevertheless, Valve is pragmatic with respect to hierarchy:
“Valve is not averse to all organizational structure—it crops up in many forms all the time, temporarily. But problems show up when hierarchy or codified divisions of labor either haven’t been created by the group’s members or when those structures persist for long periods of time. We believe those structures inevitably begin to serve their own needs rather than those of Valve’s customers. The hierarchy will begin to reinforce its own structure by hiring people who fit its shape, adding people to fill subordinate support roles. Its members are also incented to engage in rent-seeking behaviors that take advantage of the power structure rather than focusing on simply delivering value to customers.” – The Valve employee handbook
Whether Valve knows it or not, their success is due to their respect of some of Gall’s system laws:
- Systems develop goals of their own as soon as they come into existence – and intra-system goals come first.
- Loose systems last longer and work better. Efficient systems are dangerous to themselves and others.
Tainted Themes
In “The Collapse of Complex Societies“, Joseph Tainter defines “complexity” as:
Complexity is generally understood to refer to such things as the size of a society, the number and distinctiveness of its parts, the variety of specialized social roles that it incorporates, the number of distinct social personalities present, and the variety of mechanisms for organizing these into a coherent, functioning whole. – Joseph Tainter.
Joseph then defines “collapse” as:
A society has collapsed when it displays a rapid, significant loss of an established level of sociopolitical complexity. Collapse then is not a fall to some primordial chaos, but a return to the normal human condition of lower complexity. – Joseph Tainter
Mr. Tainter then surveys the landscape of historic, archaeological, and anthropological literature explaining the collapse of societies like the western Roman empire, the Mayans, the Hittites, the Mycenaeans, etc. He groups the theories of collapse into 11 “tainted” themes:
Joseph then skillfully makes a case that all these specialized themes can be subsumed under his simple and universal theory of collapse:
His theory goes something like this. As a society grows, it necessarily becomes more complex. Exceedingly more and more investment in complexity (infrastructure, basic services, defense, food production control, public tributes to the elite to maintain their legitimacy in the minds of the non-elites) is then required to hold the society together. However, as the graphic below shows, at some point in time, the marginal return on the investments in complexity reaches a tipping point at which the society becomes vulnerable to collapse from one or more of the subsumed themes.
To illustrate further, BD00 presents the dorky growth-to-collapse scenario below:
As shown, societal growth begets a larger and more internally diverse production subsystem. That same growth requires investment in more and varied control (Ashby’s law of requisite variety) over production so that “the center can hold” and the society can “retain its identity as a whole“. In this runaway positive feedback system, the growing army of controller layers siphons off more and more of the production outputs for itself – starving the production subsystem in the process.
To prevent the production subsystem from dispersing (or revolting) and keep the whole system growing, more and more investment is poured into production control (compliance and efficiency) in an attempt to increase output and keep both the production and control subsystems viable. However, as the control subsystem growth outpaces production subsystem growth and a caste system emerges, the control subsystem requires a larger and larger share of the production subsystem outputs for itself – which further weakens and constrains and alienates the production subsystem. Hence, the “declining marginal return on investment in complexity” machine is kicked into overdrive and the vulnerability to collapse appears on the horizon. D’oh!
In your growing “society“, is the controller subsystem growing faster than the production subsystem? Are more specialized controller/administration layers being added faster than producers? Is the caste system becoming more stratified and prejudiced? Are more and more processes/rules/policies being imposed on the production subsystem for increased compliance and efficiency? Is the army of growing controllers siphoning off more and more of the production system outputs for themselves? If so, then maybe your society is vulnerable to sudden collapse. But then again, it may not. Tainter’s thesis is simply a bland and drama-less, economically based theory. It might be tainted itself.
Say it taint so, shoeless Joe! – unknown kid
It’s Systems As Such
BD00 is tickled pink to have discovered the work of systems guru John Gall through Ruth Malan and thinkpurpose (muchos gracias Ruth and TP!). Behold the wisdom of the man:
If you’re an anointed leader (a.k.a dictator) and/or so-called change agent, you better freakin’ paste these galling truisms on your wall and consult them before designing your next system or mandating a change “from above” to an existing system. Actually, fuggedaboud it. It doesn’t matter. You’re hosed no matter what; even if you think you aint.
The problem isn’t THE system, it’s systems as such – John Gall
The Gall Of That Man!
A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system. – John Gall (1975, p.71)
This law is essentially an argument in favour of underspecification: it can be used to explain the success of systems like the World Wide Web and Blogosphere, which grew from simple to complex systems incrementally, and the failure of systems like CORBA, which began with complex specifications. – Wikipedia
We can add the Strategic Defense Initiative (Star Wars), the FBI’s Virtual Case File System (VCS), JTRS, FCS, and prolly a boatload of other high falutin’ defense projects to the list of wreckage triggered by violations of Gall’s law. Do you have any other majestic violations you’d like to share? Can you cite any counter-examples that attempt to refute the law….
One of the great tragedies of life is the murder of a beautiful theory by a gang of brutal facts – Benjamin Franklin
C++, which started out simply as “C With Classes“, is a successful complex “system“. Java, which started out as a simple and pure object-oriented system, has evolved into a successful complex system that now includes a mix of functional and generic programming features. Linux, which started out as a simple college operating system project, has evolved into a monstrously successful complex system. DDS, which started out as a convergence of two similar, field-tested, pub-sub messaging implementations from Thales Inc. and RTI Inc., has evolved into a successful complex system (in spite of being backed by the OMG). Do you have any other law abiding citizens you’d like to share?
Gall’s law sounds like a, or thee, platform for Fred Brooks‘ “plan to throw one away” admonition and Grady Booch‘s “evolution through a series of stable intermediate forms” advice.
Here are two questions to ponder: Is your org in the process of trying to define/develop a grand system design from scratch? Scanning your project portfolio, can you definitively know if you’re about to, or currently are, attempting a frontal assault on Gall’s galling law – and would it matter if you did know?
Posing Your Way To Success
On the left-hand side of the diagram below, we have a static system design structure where four “workers” and their manager have their hearts set on producing a world class product. On the right, we have an alternate system design where only one worker has his/her heart set on creating a world class product. The other team members are too busy posing, competing, and jockeying for positional stature in the mind of the manager for their next upward career move. It’s not shown on the diagram, but that manager is also jousting with his peers to advance his/her career.
In orgs where the policies/procedures/incentives/rewards promote individual performance over (and at the expense of) team performance, you’ll most likely find the design structure on the right in operation. Which archetype does your system design map onto?
Three Degrees Of Distribution
Behold the un-credentialed and un-esteemed BD00’s taxonomy of software-intensive system complexity:
How many “M”s does the system you’re working on have? If the answer is three, should it really be two? If the answer is two, should it really be one? How do you know what number of “M”s your system design should have? When tacking on another “M” to your system design because you “have to“, what newly emergent property is the largest complexity magnifier?
Now, replace the inorganic legend at the top of the page with the following organic one and contemplate how the complexity and “success” curves are affected:
Monopolistic Providers
In search of economies of scale, centrally planned and controlled economies in nations and corporations tend to create monopolistic providers of goods and services. For example, in corporations, accounting, personnel, and R&D departments are usually deliberately organized as subsidized monopolies. They are subsidized in the sense that the users of their products or services do not pay for them directly; the supplying units are supported financially by funds that are allocated to them from above. The pool from which these funds are drawn is filled by a “tax” allocated from above to the units served. Monopolistic units that are subsidized are generally insensitive and unresponsive to the users of their services, but they are sensitive and responsive to the desires of the higher-level units that subsidize them. These higher level units are even more removed from the units served than the serving units. As a result, they are often unaware of, or unresponsive to, the needs and desires of the internal users of monopolistically provided goods and services. – Russell Ackoff (Ackoff’s Best: His Classic Writings on Management)
OK, time to practice my “bent” UML modeling skills and test your understanding with the class and sequence diagram pair below. The class diagram provides a structural view of a fictional Ackoffian system. The sequence diagram steps through an amalgam of behaviors in a world where monopolies rule. Any questions, comments, critiques, accolades, WTFs?
Why am I here?
WTF?
Meh!
D'oh!
My BTC Address
Bulldozer00's Blog 