Stacks And Icebergs
The picture below attempts to communicate the explosive growth in software infrastructure complexity that has taken place over the past few decades. The growth in the “stack” has been driven by the need for bigger, more capable, and more complex software-intensive systems required to solve commensurately growing complex social problems.
In the beginning there was relatively simple, “fixed” function hardware circuitry. Then, along came the programmable CPU (Central Processing Unit). Next, the need to program these CPUs to do “good things” led to the creation of “application software”. Moving forward, “operating system software” entered the picture to separate the arcane details and complexity of controlling the hardware from the application-specific problem solving software. Next, in order to keep up with the pace of growing application software size , the capability for application designers to spawn application tasks (same address space) and processes (separate address spaces) was designed into the operating system software. As the need to support geographically disperse, distributed, and heterogeneous systems appeared, “communication middleware software” was developed. On and on we go as the arrow of time lurches forward.
As hardware complexity and capability continues to grow rapidly, the complexity and size of the software stack also grows rapidly in an attempt to keep pace. The ability of the human mind (which takes eons to evolve compared to the rate of technology change) to comprehend and manage this complexity has been overwhelmed by the pace of advancement in hardware and software infrastructure technology growth.
Thus, in order to appear “infallibly in control” and to avoid the hard work of “understanding” (which requires diligent study and knowledge acquisition), bozo managers tend to trivialize the development of software-intensive systems. To self-medicate against the pain of personal growth and development, these jokers tend to think of computer systems as simple black boxes. They camouflage their incompetence by pretending to have a “high level” of understanding. Because of this aversion to real work, these dudes have no problem committing their corpocracies to ridiculous and unattainable schedules in order to win “the business”. Have you ever heard the phrase “aggressive schedule”?
“You have to know a lot to be of help. It’s slow and tedious. You don’t have to know much to cause harm. It’s fast and instinctive.” – Rudolph Starkermann
Leave a comment
Why am I here?
WTF?
Meh!
D'oh!
My BTC Address
Bulldozer00's Blog 
A very true and insightful article into the real world issues most geeks face working with non-geeks.
I think you have been there and done that. An extremely in-depth understanding is hinted at here. Wish you will expand on this theme.
My experience has been “creating good systems take a good amount of time, irrespective of other factors”, especially for systems that have too many parts and comprehensive complexity. No one feels this more than the sincere person trying to build one => which by some kind of reverse logic proves that most people are insincere about building good durable reliable software systems.
Missed one level where the application rode on the hardware its self the OS was a make your own. Software Designers are doing it to them selves too. When faced with schedule and budget concerns the software designers will down play the complexity.
Or there will be the silver bullet design tool that will automate the process and push the button for code development.
For the 25+ years of software development I have been involved in the complexity has been increased in what software does but not the recognition of it.
Also we now have so much complexity in the what makes up a software system the lone-wolf programmer that would fix the software at the end of development cycle has trouble getting to work alone.
Hi Ray,
Good point about “rolling your own” OS’s. It’s also true for the “Middleware” layer. Nevertheless, I don’t think that the message of the blarticle suffers too much from this COTS-vs-Custom omission. I did, however, miss the rise of application “Frameworks”, rolled or not.