Archive
The Hidden Preservation State
In the software industry, “maintenance” (like “system“) is an overloaded and overused catchall word. Nevertheless, in “Object-Oriented Analysis and Design with Applications“, Grady Booch valiantly tries to resolve the open-endedness of the term:
It is maintenance when we correct errors; it is evolution when we respond to changing requirements; it is preservation when we continue to use extraordinary means to keep an ancient and decaying piece of software in operation (lol! – BD00) – Grady Booch et al
Based on Mr. Booch’s distinctions, the UML state machine diagram below attempts to model the dynamic status of a software-intensive system during its lifetime.
The trouble with a boatload of orgs is that their mental model and operating behavior can be represented as thus:
What’s missing from this crippled state machine? Why, it’s the “Preservation” state silly. In reality, it’s lurking ominously behind the scenes as the elephant in the room, but since it’s hidden from view to the unaware org, it only comes to the fore when a crisis occurs and it can’t be denied any longer. D’oh! Call in the firefighters.
Maybe that’s why Mr. “Happy Dance” Booch also sez:
… reality suggests that an inordinate percentage of software development resources are spent on software preservation.
For orgs that operate in accordance to the crippled state machine, “Preservation” is an expensive, time consuming, resource sucking sink. I hate when that happens.
The BD00 Process
I believe that human beings love personal stories and direct experiential reports. Thus, even though it’s highly proprietary and super secret, I’m going to expose the somewhat repeatable “process” that I use for whipping up dumbass blog posts.
Here’s how BD00 does the dirty deed…
- Every morning, after going to bed between 8-9 PM, I wake up between 4-5 AM.
- I fire up a pot of coffee and then sit down at the computer.
- I navigate to the wordpress.com BD00 dashboard.
- I mine “fieldstone” writing ideas from: my “gym notes“, my quotes pages, an interesting interaction or tainted observation at work, random web surfing, my kindle highlights, a twitter post.
- I wait…..
- A freakin’ miracle occurs!
- I start writing words or drawing a picture, or both.
- I chaotically jump back and forth ‘tween writing and drawing – iteration city.
- Sometimes, I write some words, draw a picture, and then finish the words.
- Sometimes, I write all the words first and then re-scan them for drawing ideas.
- Sometimes, I draw a partial picture, write some words, and then I finish the pic.
- Sometimes, I draw a complete picture first and then I write the words that go with it.
- I iterate over the words + picture combo multiple times until…. I say “WTF, I’m done!“
Of course, even though the BD00 process is expressed as a nice and tidy bulleted list above, it’s not a procedure. To highlight the non-sequential nature of the process, here’s the UML state machine diagram model:
Note the numerous initial entry points and that every state has an iterative transition “to self“. That’s because I Am A Strange Loop.
Attack And Advance
Check out this recent Grady Booch tweet on the topic of “why” he, James Rumbaugh, and Ivar Jacobson syntegrated their individual modeling technologies into what became the UML standard:
Over 20+ years ago, when the rate of global change was just a mere inkling of what it is today, my friend Bill Livingston stated in HFAW that: “complexity has a bright future“. He was right on the money and I admire people like Bill and the three UML amigos for attacking complexity (a.k.a ambiguity) head-on – especially when lots of people seem to love doing the exact opposite – piling complexity on top of complexity.
Extreme complexity may not be vanquishable, but (I think) it can be made manageable with the application of abstract “systems thinking” styles and concrete tools/techniques/processes created by smart and passionate people, no?
Scrumming For Dollars
“Systems Engineering with SysML/UML” author Tim Weilkiens recently tweeted:
Tim’s right. Check it out yourself: Scrum Guide – 2011.
Before Tim’s tweet, I didn’t know that “software” wasn’t mentioned in the guide. Initially, I was surprised, but upon further reflection, the tactic makes sense. Scrum creators Ken Schwaber and Jeff Sutherland intentionally left it out because they want to maximize the market for their consulting and training services. Good for them.
As a byproduct of synthesizing this post, I hacked together a UML class diagram of the Scrum system and I wanted to share it. Whatcha think? A useful model? Errors, omissions? Does it look like Scrum can be applied to the development of non-software products?
Product Lifetime
The UML state transition diagram below depicts the growth, maturity, and retirement of a large, software-intensive product. There are a bazillion known and unknown factors that influence the longevity and profitability of a system, but the greatest determinant is how effectively the work in the upfront “Develop” state is executed.
If the “Develop” state is executed poorly (underfunded, undermanned, mis-manned, rushed, “pens down“, etc), then you better hope you’ve got the customer by the balls. If not, you’re gonna spend most of your time transitioning into, and dwelling within, the resource-sucking “Fix” state. If you do have cajones in hand, then you can make the customer(s) pay for the fixes. If you don’t, then you’re hosed. (I hate when that happens.)
If you’ve done a great job in the “Develop” state, then you’re gonna spend most of your time transitioning into and out of the “Enhance” state – keeping your customer delighted and enjoying a steady stream of legitimate revenue. (I love when that happens.)
The challenge is: While you’re in the “Develop” state, how the freak do you know you’re doing a great job of forging a joyous and profitable future? Does being “on schedule” and “under budget” tell you this? Do “checked off requirements/design/code/test reviews” tell you this? Does tracking “earned value” metrics tell you this? What does tell you this? Is it quantifiably measurable?
The Universal Process Of Personal Experience
Yesterday’s post revealed to the world the static structure of the system in our head that brings personal experience to life. In the scoop of a lifetime (Geraldo Rivera eat your heart out), the system elements (mind, ego, thoughts, feelings) and the relationships between them were unmasked and exposed for all to marvel at. Woot!
Today, on the day we camp out at the feed trough and give thanks, we’ll explore the mysterious dynamics involved in this system design from the divine. Lo and behold… the universal process that creates personal experience from nothingness:
Via a yet undiscovered secret global WiMax communication system that uses quantum tunneling for zero latency source-to-sink transmission, impersonal thoughts (conjured up in the sole Thought Factory located in China) are manufactured and coupled to the ether. The mechanistic brain then serves as a receiver of thoughts and the source of fuel for the personal Ego.
Next up in the zero latency pipeline is the Ego (a.k.a the “little” me). The Ego analyzes, interprets, gives meaning to, and binds feelings to each received thought that it decides to accept. These “I” thoughts are then injected into the Mind and, voila, personal experience is manifest!
So that’s it folks. Rejoice! You don’t have to go to church anymore and you can call off the search for enlightenment . By using BD00 as a conduit from the unknown to the known, the universe has revealed all.
Spiritual UML
Because he is the chosen one, the universe spaketh to BD00 last night: “Go forth my son, and employ the UML to teach the masses the true nature of the mind!“. Fearful of being annihilated if he didn’t comply, BD00 sat down and waited for an infusion of cosmic power to infiltrate his being (to catalyze the process, BD00 primed the pump with a three olive dirty ‘tini and hoped the universe didn’t notice).
With mouse in trembling hand and an empty Visio canvas in front of him, BD00 waited…. and waited… and waited. Then suddenly, in mysterious Ouja board fashion, the mouse started moving and clicking away. Click, click, click, click.
After exactly 666 seconds, “revision 0” was 90% done. The secrets of the metaphysical that have eluded the best and brightest over the ages were captured and revealed in the realm of the physical! Lo and behold….. the ultimate UML class diagram:
Of course, the “Thought Factory” class is located in China. It efficiently and continuously creates (at rock bottom labor costs) every thought that comes/stays/goes through each of the 7 billion living brains on earth.
Manager Types II
This post is an updated refinement of BD00’s class hierarchy for the manager types previously presented in the UCBH post. For your viewing displeasure, I’ve reproduced the “rev 0” version of the inheritance tree here:
The “rev 1” version, with all class operations elided because they’re not important for understanding the message I want to get across, is shown below. The absence of the “Tweener” in rev 0, which inherits the attributes and operations from both the “Bozo” and “Helper” classes, was a major mistake.
“Rev 1” is a much more accurate mental model of the manager kingdom because, as the probability density function below shows, the vast majority of manager “objects” are of the relatively boring, harmless, and ho-hum “Tweener” type.
If you look closely at the threshold locations in the scraggly drawn probability distribution, BD00 has postulated that even though the population is comprised mostly of “Tweeners“, there are more BMs than PHORs. Do you agree?
Luckily and happily, BD00 has never worked for, or with, a conscious BM. But he’s directly heard, and indirectly read, several stories from those poor souls who have (are you one of them?). Thus, BD00 is convinced that they do exist in nature.
All models are wrong. Some, however, are useful – George Box
DataLoggerThread
The figure below models a program in which a pipeline of worker threads communicate with each other via message passing. The accordion thingies ‘tween the threads are message queues that keep the threads loosely coupled and prevent message bursts from overwhelming downstream threads.
During the process of writing one of these multi-threaded programs to handle bursty, high rate, message streams, I needed a way to periodically extract state information from each thread so that I could “see” and evaluate what the hell was happening inside the system during runtime. Thus, I wrote a generic “Data Logger” thread and added periodic state reporting functionality to each worker thread to round out the system:
Because the reporting frequency is low (it’s configurable for each worker thread and the default value is once every 5 seconds) and the state report messages are small, I didn’t feel the need to provide a queue per worker thread – YAGNI.
The figure below shows a more detailed design model of the data logging facility in the form of a “bent” UML class diagram. Upon construction, each DataLoggerThread object can be configured to output state messages to a user named disk file and/or the global console during runtime. The rate at which a DataLoggerThread object “pops” state report messages from its input queue is also configurable.
The DataLoggerThread class provides two different methods of access to user code at runtime:
void DataLoggerThread::record_txt_block(const Data&)
and
void DataLoggerThread::operator<<(const Data&).
Objects of the DataLoggerThread class run in their own thread of execution – transparently in the background to mainline user code. On construction, each object instance creates a mutex-protected, inter-thread queue and auto-starts its own thread of operation behind the scenes. On destruction, the object gracefully self-terminates. During runtime, each DataLoggerThread object polls its input queue and formats/writes the queue entries to the global console (which is protected from simultaneous, multiple thread access by a previously developed CoutMonitor class) and/or to a user-named disk log file. The queue is drained of all entries on each (configurable,) periodic activation by the underlying (Boost) threads library.
DataLoggerThread objects pre-pend a “milliseconds since midnight” timestamp to each log entry just prior to being pushed onto the queue and a date-time stamp is pre-pended to each user supplied filespec so that file access collisions don’t occur between multiple instances of the class.
That’s all I’m gonna disclose for now, but that’s OK because every programmer who writes soft, real-time, multi-threaded code has their own homegrown contraption, no?
“As-Built” Vs “Build-To”
The figure below shows one (way too common) approach for developing computer programs. From an “understanding” in your head, you just dive into the coding state and stay there until the program is “done“. When it finally is “done“: 1) you load the code into a reverse engineering tool, 2) press a button, and voila, 3) your program “As-Built” documentation is generated.
For trivial programs where you can hold the entire design in your head, this technique can be efficient and it can work quite well. However, for non-trivial programs, it can easily lead to unmaintainable BBoMs. The problem is that the design is “buried” in the code until after the fact – when it is finally exposed for scrutiny via the auto-generated “as-built” documentation.
With a dumb-ass reverse engineering tool that doesn’t “understand” context or what the pain points in a design are, the auto-generated documentation is often overly detailed, unintelligible, camouflage in which a reviewer/maintainer can’t see the forest for the trees. But hey, you can happily tick off the documentation item on your process checklist.
Two alternative, paygo types of development approaches are shown below. During development, the “build-to” design documentation and the code are cohesively produced manually. Only the important design constructs are recorded so that they aren’t buried in a mass of detail and they can be scrutinized/reviewed/error-corrected in real-time during development – not just after the fact.
I find that I learn more from the act of doing the documentation than from pushing an “auto-generate” button after the fact. During the effort, the documentation often speaks to me – “there’s something wrong with the design here, fix me“.
Design is an intimate act of communication between the creator and the created – Unknown
Of course, for developers, especially one dimensional extreme agilista types who have no desire to “do documentation” or learn UML, the emergence of reverse engineering tools has been a Godsend. Bummer for the programmer, the org he/she works for, the customer, and the code.
Why am I here?
WTF?
Meh!
D'oh!
My BTC Address
Bulldozer00's Blog 