Archive
So Professional That It’s Unprofessional
Mike Williams is one of the big three Erlang creators (along with Joe Armstrong and Robert Virding) and the developer of the first Erlang Virtual Machine. Since then, he’s “moved up” and has been working as a software engineering manager for 20+ years. In his InfoQ talk titled “The Ideal Programmer – Why They Don’t Exist and How to Manage Without Them?“, Mike presents this hilarious slide:
It’s hilarious because, if you browse web sites like LinkedIn.com and Monster.com, you’ll find tons of similar, impossible-to-satisfy job descriptions. Everybody, especially the job description writer, knows that exhaustive “requirements” lists like these are a crock of BS. This practice is so professional that it’s unprofessional. So, why does it persist?
The Undiscussable Issues Of Code Reviews
Everyone and their brother, including BD00, thinks source code inspections are a best practice for detecting and eliminating defects before they burrow into your code to wreak havoc during expensive, downstream debugging debacles. There are reams of study data to support the efficiency of code reviews (relative to testing) in the battle of the bug. But, the world being as messy as it is, there’s a fly in the ointment. That fly is the cost required to, and this is important, “effectively” execute code reviews:
“And yet, at somewhere around 100 lines of code inspected per hour, it takes an investment of many hours to inspect even the smallest piece of software, say a couple of thousand of lines of code.” – Facts And Fallacies Of SW Engineering, Robert Glass, P106
“There’s a sweet spot around 150 lines per hour that detects most of the bugs you’re going to find.” – The Art Of Designing Embedded Systems, Jack Gannsle, P21
Given these metrics, let’s concoct a concrete example so we can further investigate the cost issue :
Assuming the simple math is correct, the project manager in this example should “explicitly” allocate $200K and 4 calendar months of effort somewhere in his/her micro-managed Microsoft Project plan for our 100K line program, no?
The question is, when was the last time you saw a project plan that transparently and explicitly enumerated the time and cost for performing those code reviews mandated by your CMMI L3+ compliant process? If you haven’t seen such a plan, find out what kind of blowback you get from asking to have the resources required for “effective” code reviews embedded in your current or next prescriptive project plan. For even more drama, ask your QA group if there are any metrics (bugs found/review-hour, bug densities) that indicate how effective your code reviews are.
Since we’re having fun here, let’s go a little deeper into this undiscussable underworld by examining these two assumptions from our fabricated example:
- The reviewers are qualified and readily available: they understand the programming language(s) the code is written in; they have memorized the 200 page home-grown coding standards manual(s); they understand and have memorized the functional requirements the code is supposed to implement.
- The reviews are synchronized; the reviewers all start and end the code review at the same time.
Are these assumptions realistic? If not, then how do the budget and schedule numbers get affected?
Finally, in an iterative and incremental development process, when should code reviews take place? Piece-meal and disruptively after checking in each increment’s unit-tested code? In one big, formal she-bang after all the unit tests have passed?
How do you do code reviews, and how effective are they?
Firefighter Fran
Being anointed as a firefighter in an org that’s constantly battling blazes is one of the highest callings there is for any group member not occupying a coveted slot in the chief’s inner circle. Hell, since you didn’t start the fire and you’re gonna try your best to save the lot from a financial disaster, it’s a can’t-lose situation. If you fail, you’ll be patted on the back with a “nice try soldier; now we’re gonna go find the firestarter and kick his arse to kingdom come”. If you extinguish the blaze, at least you’ll lock in that 2% raise for this fiscal year. You might even get an accompanying $25 Dunkin Donuts gift card to keep your spare tire inflated with dozens of delectable, confectionary delights.
Given the above context, let’s start this heart-warming story off with you in the glorious role of firefighter Fran. You, my dear Fanny, oops, I mean Franny, have been assigned to put out a major blaze in one of your flagship legacy software products before it spreads to one of the nearby money silos and blows it sky high. Time, as always, is of the utmost importance. D’oh!
Since the burning pile of poop’s “agile” architecture and design documentation is a bunch of fluffy camouflage created solely to satisfy some process compliance checklist, you check out the code base and directly fire up vi (IDEs are for new age pussies!) for some serious sleuthing.
After glancing at the source tree‘s folder structure and concluding that it’s an incomprehensible, acronym-laden quagmire, you take the random plunge. With fingers a tremblin’ and fire hose a danglin’, you open up one of the 5000 source code files that’s postfixed with the word “Main“. You then start sequentially reading some hieroglyphics that’s supposed to be source code and you come to an abrupt halt when you see this:
And… that’s it! The story pauses here because BD00’s lizard brain is about to explode and it’s your turn to provide some “collaborative“, creative input.
So, what does our guaranteed-to-be-hero, fireman Fran, do next? Does the fire get doused? Does the pecuniary silo explode? Is the firestarter ever found? Does Fanny get the DD gift card? If so, how many crullers and donut holes does he scarf down?
Who knows, maybe you can become a self-proclaimed l’artiste like BD00 too, no?
Patience Grasshoppa, Patience
It won’t arrive for awhile, but it’s on its way: the C++11 rendition of Bjarne Stroustrup‘s “The C++ Programming Language“.
In case you didn’t already know, the C++11 editions of these tomes are already available for your consumption:
But of course, you did already did know this. There are gifted copies of each of these references on your and your colleague’s desks because your company cares about you, your productivity growth, and continuously enhancing the quality of its revenue-generating product portfolio. That’s why it obsesses over little, but simultaneously huge, things like this for you, its products, and its future well being. It’s a quiet, low-key, low-cost, win-win-win situation that makes you feel warm and fuzzy each day you walk in the door to joyfully add your contribution to the whole.
And puh-leeze, no comments from the hip and cool Ruby, Python, Java, Go, Scala, C#, etc, peanut galleries. On this bogus blawg, only pope BD00 is approved to promote or trash other programming languages, ideas, groups, concepts, institutions, traditions. Well, you get the idea.
Note: I created the above image at says-it.com. It’s a wonderful site, so hop on over there and give it a whirl.
Beginner, Intermediate, Expert
Check out this insightful quote from Couchbase CTO Damien Katz:
Threads are hard because, despite being extremely careful, it’s ridiculously easy to code in hard to find, undeterministic, data races and/or deadlocks. That’s why I always model my multi-threaded programs (using the MML, of course) to some extent before I dive into code:
Note that even though I created and evolved (using paygo, of course) the above, one page “agile model” for a program I wrote, I still ended up with an infrequently occurring data race that took months, yes months, to freakin’ find. The culprit ended up being a data race on the (supposedly) thread-safe DB2 data structure accessed by the AT4, AT6, and AT7 threads. D’oh!
Great, But Weird And Slow
Damien Katz is the soft-spoken Couchbase CTO and creator of the Apache CouchDB (written in Erlang and C/C++). In this InfoQ presentation, “Erlang, the Language from the Future?“, Damien praises the Erlang programming language. But he also points out its turds: weirdness and slowness.
Damien says that Erlang’s weird Prolog-like syntax prevents massive adoption; which prevents lack of massive investment; which prevents massive VM performance improvement.
To prove his point, Mr. Katz uses Java as an example of “getting it right“. When Java arrived on the scene in the mid-90’s, it was dirt slow compared to C and C++. But because Java’s syntax was C-like and it didn’t suffer from the frustrating quirks and gotchas of C/C++, it was embraced quickly and massively by the software industry. This lead to massive investment, which lead to large JVM performance increases and a boatload of productivity-enhancing development tools.
I like Erlang a lot for the concurrent, distributed type of software I write. However, Damian hits the nail right on the head. What do you think?
Best Actor Award
I recently watched (Trifork CTO and Erjang developer) Kresten Krab Thorup give this terrific talk: “Erlang, The Road Movie“. In his presentation, Kresten suggested that the 20+ year reign of the “objects” programming paradigm is sloooowly yielding to the next big problem-solving paradigm: autonomous “actors“. Using Thomas Kuhn‘s well known paradigm-change framework, he presented this slide (which was slightly augmented by BD00):
Kresten opined that the internet catapulted Java to the top of the server-side programming world in the 90s. However, the new problems posed by multi-core, cloud computing, and the increasing need for scalability and fault-tolerance will displace OOP/Java with actor-based languages like Erlang. Erlang has the upper-hand because it’s been evolved and battle-tested for over 20 years. It’s patiently waiting in the wings.
The slide below implies that the methods of OOP-based languages designed to handle post-2000 concurrency and scalability problems are rickety graft-ons; whereas the features and behaviors required to wrestle them into submission are seamlessly baked-in to Erlang’s core and/or its OTP library.
So, what do you think? Is Mr. Thorup’s vision for the future direction of programming correct? Is the paradigm shift underway? If not, what will displace the “object” mindset in the future. Surely, something will, no?
Too much of my Java programs are boilerplate code. – Kresten Krab Thorup
Too much of my C++ code is boilerplate code. – Bulldozer00
Java either steps up, or something else will. – Cameron Purdy
Reasonable Debugging
In Rich Hickey‘s QCon talk, “Simple Made Easy”, he hoisted this slide:
So, what can enhance one’s ability to “reason about” a program, especially a big, multi-threaded, multi-processing beast that maps onto a heterogeneous hodge-podge network of hardware and operating systems? Obviously, a stellar memory helps, but come on, how many human beings can remember enough detail in a >100K line code base to be able to debug field turds effectively and efficiently?
How about simplicity of design structure (whatever that means)? How about the deliberate and intentional use of a small set of nested, recurring patterns of interaction – both of the GoF kind and/or application specific ones? Or, shhhh, don’t say it too loudly, how about a set of layered blueprints that allow you and others to mentally “fly” over the software quickly at different levels of detail and from different aspect angles; without having to slodge through reams of “flat” code?
Do you, your managers, and/or your colleagues value and celebrate: simplicity of design structure; use of a small set of patterns of interaction; use of a set of blueprints? Do you and they walk the talk? If not, then why not? If so, then good for you, your org, your colleagues, your customers, and your shareholders.
Misapplication Of Partially Mastered Ideas
Because the time investment required to become proficient with a new, complex, and powerful technology tool can be quite large, the decision to design C++ as a superset of C was not only a boon to the language’s uptake, but a boon to commercial companies too – most of whom developed their product software in C at the time of C++’s introduction. Bjarne Stroustrup‘s decision killed those two birds with one stone because C++ allowed a gradual transition from the well known C procedural style of programming to three new, up-and-coming mainstream styles at the time: object-oriented, generic, and abstract data types. As Mr. Stroustrup says in D&E:
Companies simply can’t afford to have significant numbers of programmers unproductive while they are learning a new language. Nor can they afford projects that fail because programmers over enthusiastically misapply partially mastered new ideas.
That last sentence in Bjarne’s quote doesn’t just apply to programming languages, but to big and powerful libraries of functionality available for a language too. It’s one challenge to understand and master a language’s technical details and idioms, but another to learn network programming APIs (CORBA, DDS, JMS, etc), XML APIs, SQL APIs, GUI APIs, concurrency APIs, security APIs, etc. Thus, the investment dilemma continues:
I can’t afford to continuously train my programming workforce, but if I don’t, they’ll unwittingly implement features as mini booby traps in half-learned technologies that will cause my maintenance costs to skyrocket.
BD00 maintains that most companies aren’t even aware of this ongoing dilemma – which gets worse as the complexity and diversity of their product portfolio rises. Because of this innocent, but real, ignorance:
- they don’t design and implement continuous training plans for targeted technologies,
- they don’t actively control which technologies get introduced “through the back door” and get baked into their products’ infrastructure; receiving in return a cacophony of duplicated ways of implementing the same feature in different code bases.
- their software maintenance costs keep rising and they have no idea why; or they attribute the rise to insignificant causes and “fix” the wrong problems.
I hate when that happens. Don’t you?
Highly Available Systems == Scalable Systems
In this QCon talk: “Building Highly Available Systems In Erlang“, Erlang programming language creator and highly-entertaining speaker Joe Armstrong asserts that if you build a highly available software system, then scalability comes along for “free” with that system. Say what? At first, I wanted to ask Joe what he was smoking, but after reflecting on his assertion and his supporting evidence, I think he’s right.
In his inimitable presentation, Joe postulates that there are 6 properties of Highly Available (HA) systems:
- Isolation (of modules from each other – a module crash can’t crash other modules).
- Concurrency (need at least two computers in the system so that when one crashes, you can fix it while the redundant one keeps on truckin’).
- Failure Detection (in order to fix a failure at its point of origin, you gotta be able to detect it first)
- Fault Identification (need post-failure info that allows you to zero-in on the cause and fix it quickly)
- Live Code Upgrade (for zero downtime, need to be able to hot-swap in code for either evolution or bug fixes)
- Stable Storage (multiple copies of data; distribution to avoid a single point of failure)
By design, all 6 HA rules are directly supported within the Erlang language. No, not in external libraries/frameworks/toolkits, but in the language itself:
- Isolation: Erlang processes are isolated from each other by the VM (Virtual Machine); one process cannot damage another and processes have no shared memory (look, no locks mom!).
- Concurrency: All spawned Erlang processes run in parallel – either virtually on one CPU, or really, on multiple cores and processor nodes.
- Failure Detection: Erlang processes can tell the VM that it wants to detect failures in those processes it spawns. When a parent process spawns a child process, in one line of code it can “link” to the child and be auto-notified by the VM of a crash.
- Fault Identification: In Erlang (out of band) error signals containing error descriptors are propagated to linked parent processes during runtime.
- Live Code Upgrade: Erlang application code can be modified in real-time as it runs (no, it’s not magic!)
- Stable Storage: Erlang contains a highly configurable, comically named database called “mnesia” .
The punch line is that systems composed of entities that are isolated (property 1) and concurrently runnable (property 2) are automatically scalable. Agree?
Why am I here?
WTF?
Meh!
D'oh!
My BTC Address
Bulldozer00's Blog 