Archive
Performance Playground
Since I work on real-time software projects where tens of thousands of data samples per second must be filtered, manipulated, and transformed into higher level decision-support information, performance in the main processing pipeline is important. If the software can’t keep up with the unrelenting onslaught of data streaming in from the “real world“, internal buffers/queues will overflow at best and the system will crash at worst. D’oh!
Because of the elevated importance of efficiency in real-time systems, I always keep a simple (one source code file) project named “performance_playground” open in my Eclipse IDE for algorithm/idiom/pattern prototyping and performance measurement. I use it to measure and optimize the performance of “chunks” of critical logic and to pit two or more candidates against each other in performance death matches. For each experiment I “branch” off of the project trunk and then I tag and commit the instantiation to archive the results.
The source code for the performance_playground project is shown below. The program’s sole external dependency is on the boost.date_time library for its platform-independent timestamping. Surely, you have the boost library set installed on all your development platforms, right?
How about you? Do you have something similar? Do you assume that all performance testing and algorithm vetting falls into the dreaded, time-wasting, “premature optimization” anti-pattern?
Avoiding The Big Ball Of Mud
Like other industries, the software industry is rife with funny and quirky language terms. One of my favorites is “Big Ball of Mud“, or BBoM. A BBoM is a mess of a software system that starts off soft and malleable but turns brittle and unstable over time. Inevitably, it hardens like a ball of mud baking in the sun; ready to crumble and wreak havoc on the stakeholder community when disturbed. D’oh!
“And they looked upon the SW and saw that it was good. But they just had to add one more feature…”
Joseph Yoder, one of the co-creators of “BBoM” along with Brian Foote, recently gave a talk titled “Big Balls of Mud in Agile Development: Can We Avoid Them?“. Out of curiosity and the desire to learn more, I watched a video of the talk at InfoQ. In his talk, Mr. Yoder listed these agile tenets as “possible” BBoM promoters:
- Lack of upfront design (i.e. BDUF)
- Embrace late changes to requirements
- Continuous evolving architecture
- Piecemeal growth
- Focus on (agile) process instead of architecture
- Working code is the one true measure of success
For big software systems, steadfastly adhering to these process principles can hatch a BBoM just as skillfully as following a sequential and prescriptive waterfall process. It’ll just get you to the state of poverty that always accompanies a BBoM much quicker.
Unlike application layer code, infrastructure code should not be expected to change often. Even small architectural changes can have far reaching negative effects on all the value-added application code that relies on the structure and general functionality provided under the covers. If you look hard at Joe’s list, any one of his bullets can insidiously steer a team off the success profile below – and lead the team straight to BBoM hell.
Happressed
Ray (the “singularity” dude) Kurzweil‘s web site has summarized Science magazine‘s breakthrough of the year: the world’s first quantum machine. The gizmo is a tiny, but visible to the naked eye, metal “paddle” whose vibrational frequency can be controlled.
No big thing right? Well, here’s what some rocket scientists from UC Santa Barbara achieved:
First, they cooled the paddle until it reached its “ground state,” or the lowest energy state permitted by the laws of quantum mechanics (a goal long-sought by physicists). Then they raised the widget’s energy by a single quantum to produce a purely quantum-mechanical state of motion. They even managed to put the gadget in both states at once, so that it literally vibrated a little and a lot at the same time—a bizarre phenomenon allowed by the weird rules of quantum mechanics.
Wild and exciting stuff, no? A macro object composed of a collection of atoms (not just some singular, invisible, sub-atomic particle) operating in two supposedly mutually exclusive states at once. Just think of any point of reference on the paddle. The scientists proved (<- this is a key word as you’ll see below) that they could force it appear in two places at once. It’s sort of like being both ecstatically happy and depressed at the same time?
When I read about this breakthrough, I frantically searched the web for a video of the event. Hell, I want to actually see and experience what it looks like. Sadly, I was disappointed. I found this clip from Gizmodo that explains the absence of any visual evidence:
“It’s important to realize that they didn’t actually observe it in a superposition. All they said was that the paddle is large enough that it could be observed by the naked eye, but not while it is in a superposition state.”
D’oh! Upon reflection, this makes sense to me based on my layman’s knowledge of quantum physics. Prior to observation by, uh, an observer, so-called reality exists as a superposition of an infinite set of continuous matter waves that can be modeled by Erwin Schrodinger‘s unassailable wave equation. At the moment of observation, poof!, an observer “collapses” the wavefunction into a singularity – in effect creating his/her own reality. The mysterious question to me is: “Why do most people seem to see essentially the same reality when they observe an object at the same point in space and time?“. After all, what are the chances that my collapsed wavefunction will coincide with yours?
Money, Tools, Materials, Know-How, Products
Since you’ve stopped by, check out the company-centered, system loop diagram below. The model is composed of three interdependent parts: the company, the customers, and the suppliers. In equilibrium, when all is well:
- The company produces products that are paid for, and consumed by, customers.
- The company purchases tools and materials from suppliers.
- The company uses the tools, materials, and the knowledge of its workforce to create the products that sustain the viability of the company.
- The company’s cash inflow exceeds it’s cash outflow
Note that if any of the links in the system get severed, the company collapses. If the products stop flowing out of the company, the life-giving cash stops flowing in. If the cash stops flowing in, the products stop flowing out. If the tools and materials stop flowing in, the products stop flowing out. From the company’s perspective, products lead to cash and cash leads to more products in a positive, self-reinforcing, feedback loop.
Using the diagram below, let’s look in more depth at the company’s cash inflows and outflows. In stable, steady-state operation, the cash inflow is managed competently by the executive team. After taking their cut of the action, the executives push some cash downward through the patriarchy to keep the operation humming and they approve of all outflows to suppliers.
What the picture doesn’t show, is the indirect source of the customer cash – the company’s product set. Lets augment the diagram above and close the loops:
There are a bazillion external, and especially, internal threats to system viability. Externally, customers can run out of cash and suppliers can go bust. Internally, bureaucratic little Hitlers, byzantine processes, lack of investments in tools and people, inequities in status and pay, silo-to-silo infighting, and poor hiring practices are among the myriad of threats that can contribute to corpo implosion. Of course, the purpose of management is to gracefully overcome the threats. But hey, regardless of whether executives and their management appointees are the cause of success or failure, they still have the right to high status and high compensation because…. well, just because.
The Curiously Recurring Scramble Pattern
It’s funny to watch software development teams hack away for months building a just barely working patch-quilt monster that they can hardly understand themselves – and then scramble at the last minute generating design documents for some big upcoming management design review or “independent” auditor dog and pony show (woof woof!).
In this Curiously Recurring Scramble Pattern (CRSP), a successful attempt to avoid the labor of thinking is made as developers frantically sprinkle Doxygen annotations throughout the code and/or load the beast into a reverse engineering tool that mechanistically generates UML diagrams to model the as-built mess. It goes without saying that the tool’s “verbose” mode is selected in order to obscure meaning and promote the illusion of high falutin’ sophistication. Of course, all of this is a waste of time (= $$$$) because the dudes doing the reviewing (self-important managers and bureaucratic auditors) don’t want to understand a thing.
When the review or audit does take place; a couple of cream puff questions and comments are bantered about, check boxes are ticked off, a couple of superficial “action items” are generated, and the whole lovefest is rubber-stamped as a great success. Whoo Hoo, we rock!
Without a doubt, you and I have never been culturally forced to participate in an instantiation of the CRSP. We are above that nonsense, right? We do something like this.
“A meeting is a refuge from the dreariness of labor and the loneliness of thought.” – Bernard Baruch
void Manager::pitchInWhereNeeded(const Work& w){}
Unless you’re a C++ programmer, you probably won’t understand the title of this post. It’s the definition of the “pitchInWhereNeeded” member function of a “Manager” class. If you look around your immediate vicinity with open eyes, that member function is most likely missing from your Manager class definition. If you’re a programmer, oops, I mean a software engineer, it’s probably also missing from your derived “SoftwareLead“, “SoftwareProjectManager“, and “SoftwareArchitect” classes too.
As the UML-annotated figure below shows, in the early twentieth century the “pitchInWhereNeeded” function was present and publicly accessible by other org objects. On revision number 1 of the “system”, as signaled by the change from “+” to “-“, its access type was changed to private. This seemingly minor change broke all existing “system” code and required all former users of the class to redesign and retest their code. D’oh!
On the second revision of the Manager class, this critical, system-trust-building member function mysteriously disappeared completely. WTF?. This rev 2 version of the code didn’t break the system, but the system’s responsiveness decreased since private self -calls by manager objects to the “pitchInWhereNeeded” function were deleted and more work was pushed back into the “other” system objects. Bummer.
Simply Notice…
I’ve been following Leo Babauta, the creator of ZenHabits.net, for several years. Check out his inspirational story of personal transformation here: “About Leo“. In a recent Zen Habits newsletter, guest poster Gail Brenner suggested the following minimalist guide to inner peace:
This list recently came to mind while I was reading some complicated and messy library code in order to understand how to incorporate its functionality into my own code mess. Surprise! The source code was too dense and it required a deeper mental stack than the shallow one in my head, so I did what I usually do in those situations – I started getting pissed off. D’oh! Then, out of nowhere, step #2 in Gail’s list came to mind. I won’t answer the questions posed in the step, but you can imagine that they weren’t positive.
So, what did I do next? I briefly looked at step 3 and then I careened off course. I went out to stalk the library code author with the intent of ripping him/her a new one. Hah, just joking! What I really did was this. I stepped back from the hundreds of lines of source code and I reverse engineered a simpler, abstracted class diagram of the mess. By distancing myself from the code and using the more abstract class diagram to understand the code, I came to the same conclusion – it was a freakin’ mess. D’oh!
Are you wondering what my next step was? I ain’t tellin’.
Yahoo! Boohoo!
Unless you were born yesterday, you’ve probably heard about the death spiral that former internet great Yahoo! has commenced. In this blarticle from TechCrunch, “Former Yahoo Engineers Shed Light On Why Delicious And Other Acquisitions Failed“, a couple of quotes from former employees brought a tear to my eye.
…it does provide a picture of a company that bogged its acquired-startups down in its company’s administrative BS. As Chad Dickerson, former Yahoo developer evangelist and the current CTO of Etsy comments, “In my experience, entrepreneurs moving into Yahoo! often got stuck doing PowerPoints about “strategy” instead of writing code and shipping products.”
Elliott-McCrea writes: I recently pulled up a worklog I was keeping in 2008-2009, and I found 18 meetings scheduled over a 9 month period discussing why Flickr’s API was poorly designed and when we’d be shutting it down and migrating it to the YOS Web Services Standard.
What I’d like to know is: “Did any of the layers of corpo honchos have any conscious clue that the patriarchical and bureaucratic monster they brought to life was killing the golden goose?” What do you think?
Leader Or Dictator?
After reading Chetan Dhruve’s “Why Your Boss Is Programmed To Be A Dictator“, I’ve had a change of heart. I’ve concluded that hierarchy is a symptom and not the dominant cause of dysfunctional corpricracies. In his book, Mr. Dhruve skillfully develops a compelling case that the lack of the right to vote managers into and out of higher status slots in the hierarchy is the real cause of poor org-wide performance and DIC-force suffering in the workplace.
Mr. Dhruve asserts that there are two canonical forms of power systems: leaderships and dictatorships. By his definition, leaders are elected into power by those they lead, and dictators assume power by any other means. In corpricracies, dictators don’t assume power by shedding blood, they assume power in a civilized manner; by anointment from higher status dictators.
The unquestioned assumption in dictatorships is that superior status equates directly with superior knowledge and judgment. In corpo dictatorships, un-submissive subjects aren’t killed. They’re marginalized at best, and fired at worst. Chetan closes his masterpiece with a brilliant quote targeted at anyone in any power structure:
If you aren’t elected, you’re a dictator – Chetan Dhruve
Quote Mangling
I love quotes because they pack a lot of energy into small packages without requiring a long winded and jargon filled explanation. I also like extending quotes to give another take on the wisdom imparted by the original text because I’ve learned to never believe anything 100 percent; including my own twisted and anguished thoughts. Here are some snarky Bulldozer00 quote enhancements:
- “When the going gets tough, the tough get going – right out the door with the cash.” (English proverb + Bulldozer00)
- “I don’t know the key to success, but the key to failure is trying to please everybody – another key to failure is trying to piss everybody off.” (Bill Cosby + Bulldozer00)
- “A lot of people mistake unnecessary complexity for sophistication – and those people are usually managers.” (Dan Ward + Bulldozer00)
- “If you are indispensable, you’re unpromotable – but at least you know how to directly create value.” (Unknown + Bulldozer00)
- “I love deadlines because I like the whooshing sound they make as they fly by – and the angst they instill in the managers who dictated them.” (Doug Adams + Bulldozer00)
- “Multitasking is not for serious work – it’s for managerial make-work.” (Bjarne Stroustrup + Bulldozer00)
- “I believe that there is such a thing as objective truth, but a lot of people have an objective truth that differs from mine – and those people are wrong.” (Cynthia Tucker+ Bulldozer00)
- “Debugging is like farting, it’s not so bad when it’s your own code – except when you crap your pants.” (Unknown + Bulldozer00)
- “What we need is more people to lose their temper in public – unless people in power are in the vicinity.” – (Watts Wacker + Bulldozer00)
- “The point of quotations is that one can use another’s words to be insulting – D’oh! and WTF?.” – (Amanda Cross+ Homer Simpson + Bulldozer00)
What quotes would you like to extend?
Why am I here?
WTF?
Meh!
D'oh!
My BTC Address
Bulldozer00's Blog 