Archive
Complexity Explosion
I’m in the process of writing a C++ program that will synthesize and inject simulated inputs into an algorithm that we need to test for viability before including it in one of our products. I’ve got over 1000 lines of code written so far, and about another 1000 to go before I can actually use it to run tests on the algorithm. Currently, the program requires 16 multi-valued control inputs to be specified by the user prior to running the simulation. The inputs define the characteristics of the simulated input stream that will be synthesized.
Even though most of the control parameters are multi-valued, assume that they are all binary-valued and can be set to either “ON” or “OFF” . It then follows that there are 2**16 = 65536 possible starting program states. When (not if) I need to add another control parameter to increase functionality, the number of states will soar to 131,072, if, and only if, the new parameter is not multi-valued. D’oh! OMG! Holy shee-ite!
Is it possible to setup the program, run the program, and evaluate its generated outputs against its inputs for each of these initial input scenarios? Never say never, but it’s not economically viable. Even if the setup and run activities can be “automated”, manually calculating the expected outputs and comparing the actual outputs against the expected outputs for each starting state is impractical. I’d need to write another program to test the program, and then write another program to test the program that tests the program that tests the first program. This recursion would go on indefinitely and errors can be made at any step of the way. Bummer.
No matter what the “experts” who don’t have to do the work themselves have to say, in programming situations like this, you can’t “automate” away the need for human thought and decision making. Based on knowledge, experience, and more importantly, fallible intuition, I have to judiciously select a handful of the 65536 starting states to run. I then have to manually calculate the outputs for each of these scenarios, which is impractical and error-prone because the state machine algorithm that processes the inputs is relatively dense and complicated itself. What I’m planning to do is visually and qualitatively scan the recorded outputs of each program run for a select few of the 65536 states that I “feel” are important. I’ll intuitively analyze the results for anomalies in relation to the 16 chosen control input values.
Got a better way that’s “practical”? I’m all ears, except for a big nose and a bald head.
“Nothing is impossible for the man who doesn’t have to do it himself.” – A. H. Weiler
Problems And Challenges
It’s easy to view a situation that requires action as a “challenge” instead of a “problem” if you don’t personally have to effect the change yourself. That’s why managers talk about challenges and workers talk about problems. Since hierarchical command and control corpocracies are inherently stratified caste systems, managers and workers don’t have a chance of seeing the same thing – a prallenge.
Four Or Two?
Assume that the figure below represents the software architecture within one of your flagship products. Also assume that each of the 6 software components are comprised of N non-trivial SLOC (Source Lines Of Code) so that the total size of the system is 6N SLOC. For the third assumption, pretend that a new, long term, adjacent market opens up for a “channel 1” subset of your product.
To address this new market’s need and increase your revenue without a ton of investment, you can choose to instantiate the new product from your flagship. As shown in the figure below, if you do that, you’ve reduced the complexity of your new product addition by 1/3 (6N to 4N SLOC) and hence, decreased the ongoing maintenance cost by at least that much (since maintainability is a non-linear function of software size).
Nevertheless, your new product offering has two unneeded software components in its design: the “Sample Distributor” and the “Multi-Channel Integrator”. Thus, if as the diagram below shows, you decide to cut out the remaining fat (the most reliable part in a system is the one that’s not there – because it’s not needed), you’ll deflate your long term software maintenance costs even further. Your new product portfolio addition would be 1/3 the original size (6N to 2N SLOC) of your flagship product.
If you had the authority, which approach would you allocate your resources to? Would you leave it to chance? Is the answer a no brainer, or not? What factors would prevent you from choosing the simpler two component solution over the four component solution? What architecture would your new customers prefer? What would your competitors do?
Best Of The Best
The breadth of variety of companies, markets, customers, industries, products, and services in the world is so wide and diverse that it can be daunting to develop objectively measurable criteria for “best in class” that cuts across all of the variability.
Being a simpleton, my pseudo-measurable criteria for a “best in class” company is:
- Everybody (except for the inevitable handful of malcontents (like me?) found in all organizations) who works in the company sincerely feels good about themselves, their co-workers, the products they build, their customers, and the company leadership.
That’s it. That’s my sole criterion (I told you I was a simpleton). Of course, the classical financial measures like year-over-year revenue growth, profitability, yada, yada, yada, matter too, but in my uncredentialed and unscholarly mind, those metrics are secondary. They’re secondary because good numbers are unsustainable unless the touchy-feely criterion is continuously satisfied.
The dilemma with any kind of “feel good” criteria is that there aren’t many good ways of measuring them. Nevertheless, one of my favorite companies, zappos.com, has conjured up a great way of doing it. Every year, CEO Tony Hsieh sends an e-mail out to all of his employees and solicits their thoughts on the Zappos culture. All the responses are then integrated and published, unedited, in a hard copy “Zappos Culture Book”.
The Zappos culture book is available free of charge to anyone who emails Tony (tony@zappos.com). Earlier this year, I e-mailed Tony and asked for a copy of the book. Lo and behold, I received the 400+ page tome, free-of-charge, four days later. I poured through the 100’s of employee, executive, and partner testimonials regarding Zappos’s actual performance against their espoused cultural values. I found no negative entries in the entire book. There were two, just two, lukewarm assessments of the company’s cultural performance. Of course, skeptics will say that the book entries were censored, and maybe they were, but I doubt it.
How would your company fare if it compiled a yearly culture book similar to Zappos’s? Would your company even entertain the idea? Would anyone feel comfortable proposing the idea? Is the concept of a culture book only applicable to consumer products companies like Zappos.com, or could its value be industry-independent?
Note: Zappos.com was recently bought out by Amazon.com. It should be interesting to see if the yearly Zappos culture book gets squashed by Jeff Bezos et al.
Man, These Guys Are Good
In general, I think that management consultants are way overpaid and full of themselves. These bozos with fat heads come waltzing in to a company in trouble and:
- analyze the “situation” from afar without getting their hands dirty,
- dispense all kinds of “proprietary” voodoo advice,
- collect their fee,
- and then bolt – leaving the ineffective corpocrats (who caused the mess in the first place) to clean up their own dung.
Notwithstanding the vitriolic diatribe in the previous paragraph, I think the following consulting dudes are the real deal: Vital Smarts. They’re people-oriented instead of mechanistically process-oriented. Collectively, they’ve talked with tens of thousands of workers; from the dweebs in the cellar to the exalted royalty in the corner of the building. They’ve also analyzed a ton of academic research to derive some down to earth, pragmatic, and potentially actionable direction for everyone – not just the patriarchs who direct the horror show. I’ve read all of their terrific bestselling books:
I’ve actually tried to employ their teachings in an attempt to be more effective in the workplace, but Ive failed miserably. Of course, it’s not their fault. It’s me and my “Unshakeable Cognitive Burden” of negativity toward all man-made command and control hierarchies.
A Professional Failure
I’m a professional failure. Why? Because I’m pretty sure that I’ve never satisfied any unreasonable schedule that I was ever “given” to meet. Since almost all schedules are unreasonable, then, by definition, I’m a professional failure. Hell, it didn’t even matter if I was the one who created the unreasonable schedule in the first place, I’ve failed. Bummer.
Looking back, I think that I’ve figured out why I underperformed (<– that’s management-speak for “failed”). It’s simply that the problem solving projects that I’ve worked on have been grossly underestimated. Why is that? Because they all required learning something new and acquiring new knowledge in the problem area of pecuniary interest.
So, how can you know if a given schedule is unreasonable, and does it matter if you conclude that meeting the schedule is a lost cause? You most likely can’t, and no, it doesn’t matter. Assume that, based on personal experience and a deep “knowing” of what’s involved in a project, you actually can determine that the schedule is a laughable, but innocent, lie. There’s nothing you can do about it. If you speak up, at best, you’ll be ignored. At worst, you’ll receive multiple peek-a-boo visits from one or more STSJs (Status Taker and Schedule Jockey) who don’t have to do any of the project work themselves.
How about you, have you been a perpetual failure like me? Of course not. Your resume says here that you have been 100% successful on every project you’ve worked on; and that implies that you’ve met every schedule. But wait, every other resume in my stack says the same thing. Damn! How am I gonna decide among all of these perfect people who gets the job?
Particular Individuals Don’t Matter
It doesn’t matter who the particular individuals in a corpocracy are. No matter how smart and well meaning they are, the awesome power of the pyramidal structure of woe to suppress their individuality and transform them into zombie clones tasked to guard the status-quo will prevail. How many of you have seen and experienced the ascension of smart, and formerly-effective, people into the ranks of the elite, only to be instantaneously transformed into ineffective druids?
Must Be An Outsider
One must be an outsider to escape being scalded for pointing out problems within a corpocracy. Unlike insiders (except for the obligatory, once a year, watered down employee survey), outsider opinions are actually solicited by the infallible hierarchs (who confidently and assuredly think they run the show). In addition, outsider pundits with “impeccable credentials” actually get paid for their analysis and recommendations! That’s why Weinberg’s “Secrets of Consulting” is in my reading queue.
Sadly, even if the situation on the left in the above diagram never happens in your org, DICs won’t stand up and expose turds that threaten the well being of the corpocracy because the image is dogmatically burned into their mind. There’s a reason why the story of the “emperor’s new clothes” is so funny and well known. The boy who pointed out his “highny”-ness’ s wardrobe malfunction was outside of the emperor’s kiss-ass court. Had he been an insider, it would have been “off with his head”.
Galileo And Kepler
To reinforce my anti-corpocracy UCB (Unshakeable Cognitive Burden), I just finished reading “The Age Of Heretics: A History Of The Radical Thinkers Who Reinvented Corporate Management“. It’s the second time in the last few months that I stumbled across the Galileo-Pope Urban story. The first time was in W. L. Livingston’s forthcoming “Design For Prevention”. Here’s a snippet from “Heretics”:
Why does Galileo Galilei have the reputation of a heretic, while his seventeenth-century fellow scientist Johannes Kepler does not? Because Kepler evaded the Church. Galileo sought to change it. The professor from Pisa spent the last third of his life arguing, with increasing fervor, that the Christian doctrines and even Bibles should be rewritten to conform to the realities he had seen through his telescope. Many of the cardinals and Church officials who censured and imprisoned him recognized the validity of the new cosmology and physics that Galileo championed, but they didn’t want to shake up their system too quickly. Too many monks and village priests clung to Ptolemy and Aristotle. The “people” would rebel at any sudden revision of the “truth.” Galileo didn’t care. Like many other heretics, past and present, he thought at first that the truth would set the institution free. He only had to show people what he had seen, and they would naturally adapt. When people doubted observations that to him were obvious, he lost his tact. He made enemies (some said needlessly) of the Jesuits, who fought bitterly to see him condemned, and he closed one of his notorious tracts, the Dialogue on the Great World Systems, with a snide lampoon of the views of Pope Urban VIII. Until then Urban had been his patron and champion. Ten months after publication in 1633, Galileo was on trial in Rome.
Here’s a snippet that is written further along in the book:
Even the Roman Catholic church eventually admitted that Galileo’s cosmology was correct—after 359 years.
Bummer. Behind the illusory cloak of modern civility, irrational and insane institutional behavior hasn’t changed much over the years. Heretics are still reviled by the bozos in power who will do whatever it takes to retain that power, and more importantly, the personal riches that automatically go along with it. Today’s well meaning but unconscious corpocrats are simply much more clever at veiling the methods that they use to annihilate heretics, even when individual heretics arise from their own ranks. Kepler rules!
Sum Ting Wong
- SYS = Systems
- SW = Software
- HW = Hardware
When the majority of SYS engineers in an org are constantly asking the HW and SW engineers how the product works, my Chinese friend would say “sum ting (is) wong”. Since they’re the “domain experts”, the SYS engineers supposedly designed and recorded the product blueprints before the box was built. They also supposedly verified that what was built is what was specified. To be fair, if no useful blueprints exist, then 2nd generation SYS engineers who are assigned by org corpocrats to maintain the product can’t be blamed for not understanding how the product works. These poor dudes have to deal with the inherited mess left behind by the sloppy and undisciplined first generation of geniuses who’ve moved on to cushy “staff” and “management” positions.
Leadership is exploring new ground while leaving trail markers for those who follow. Failing to demand that first generation product engineers leave breadcrumbs on the trail is a massive failure of leadership.
If the SYS engineers don’t know how the product works at the “white box” level of detail, then they won’t be able to efficiently solve system performance problems, or conceive of and propose continuous improvements. The net effect is that the mysterious “black box” product owns them instead of vice versa. Like an unloved child, a neglected product is perpetually unruly. It becomes a serial misbehaver and a constant source of problems for its parents; leaving them confounded and confused when problems manifest in the field.
A corpocracry with leaders that are so disconnected from the day-to-day work in the bowels of the boiler room that they don’t demand system engineering ownership of products, get what they deserve; crappy products and deteriorating financial performance.
Why am I here?
WTF?
Meh!
D'oh!
My BTC Address
Bulldozer00's Blog 