Quantification Of The Qualitative
Because he bucked the waterfall herd and advocated “agile” software development processes before the agile movement got started, I really like Tom Gilb. Via a recent Gilb tweet, I downloaded and read the notes from his “What’s Wrong With Requirements” keynote speech at the 2nd International Workshop on Requirements Analysis. My interpretation of his major point is that the lack of quantification of software qualities (you know, the “ilities”) is the major cause of requirements screwups, cost overruns, and schedule failures.
Here are some snippets from his notes that resonated with me (and hopefully you too):
- Far too much attention is paid to what the system must do (function) and far too little attention to how well it should do it (qualities) – in spite of the fact that quality improvements tend to be the major drivers for new projects.
- There is far too little systematic work and specification about the related levels of requirements. If you look at some methods and processes, all requirements are ‘at the same level’. We need to clearly document the level and the relationships between requirements.
- The problem is not that managers and software people cannot and do not quantify. They do. It is the lack of ‘quantification of the qualitative’ that is the problem.
- Most software professionals when they say ‘quality’ are only thinking of bugs (logical defects) and little else.
- There is a persistent bad habit in requirements methods and practices. We seem to specify the ‘requirement itself’, and we are finished with that specification. I think our requirement specification job might be less than 10% done with the ‘requirement itself’.
I can really relate to items 2 and 5. Expensive and revered domain specialists often do little more than linearly list requirements in the form of text “shalls”; with little supporting background information to help builders and testers clearly understand the “what” and “why” of the requirements. My cynical take on this pervasive, dysfunctional practice is that the analysts themselves often don’t understand the requirements and hence, they pursue the path of least resistance – which is to mechanically list the requirements in disconnected and incomprehensible fragments. D’oh!
Leave a comment
Why am I here?
WTF?
Meh!
D'oh!
My BTC Address
Bulldozer00's Blog 
I completely agree. Especially with the points about “multiple levels of requirements” and the over focus on “functions vs. the value the system provides to all stakeholders”.
In http://charliealfred.wordpress.com/requirements-vs-architecture/ I make the points that requirements are often assessed according to these critieria:
Clear – precise outcome, invariant or specific condition
Concise – each statement addresses one requirement – independent of the rest
Testable – observable or quantifiable outcome, reproducable
Consistent – does not conflict with other requirements
Complete – all requirements needed for suitability are specified
What I find to be interesting is that the 1st three can be assessed for individual requirement. The last two require us to understand many requirements at the same time.
In fact, I would go as far as to say you cannot evaluate consistency or completeness without a reference architecture.
(I wonder if this explains why in virtually every requirements review I’ve ever been to, the focus has been on the first three: clarity, conciseness, and testability. It’s like that old joke of the drunk looking on 12th Avenue for the keys he lost on 10th Avenue because the light is better on 12th).
Charlie
Hi Charlie,
Thanks for stopping by and sharing your insights. I like your CCTCC criteria. It may be already implied by “Testable”, but I’d also add “buildable” to the list. I find it ironic that a lot of domain experts have a tough time integrating across (in your words) “many requirements at the same time“.
I think your idea of a reference architecture, to serve as the root of reason, is a terrific one. Nevertheless, sequential thinking waterfall adherents (and they’re still around) could interpret it as a cart-before-the-horse situation.
PS – Your reqs-arch blog post is terrific.
That’s only because those “waterfall thinkers” believe that requirements are the horse. 🙂
Tom Gilb is absoluately correct on this. There are many levels of requirements. Architecture can’t come first. It has to be motivated by a clear statement of the *problem* – these are forces which are *outside* the system to be built which shape the solution. These forces take three forms:
1. Qualities – how well you can do something: utility = f(metric)
2. Constraints – limits imposted on you, like time; may be binary or scalar
3. Uncertainties – uncontrollable variation in Q or C – probability dist
So, my waterfall looks like this:
1. Understand the problem (outside the system) well
2. Use architecture and experimentation to frame a solution that works
3. Derive requirements from architecture decisions.
4. Add functional requirements, that are consistent with the architecture
5. Allocate requirements to disciplines (electrical, mechanical, software, manufacturing)
6. etc.
This approach helps ensure that the architecture is consistent with the problem, the requirements are consistent with the architecture,
and the engineering disciplines are guided by a consistent architecture and understanding of the problem.
Charlie
Once again, terrific stuff Charlie. In world class orgs, there are feedback loops triggered by learning/discovery between and across your 6 steps. They are not only “allowed”, but encouraged and facilitated.
Anthony, I love the picture of the developer being buried in a “shall”-valanche. It’s great. I’d like to reuse it someday.
Even Moses only carried 10 “shall-nots” down the mountain (well, 15 if you believe Monty Python).
Charlie
lol. Feel free to use whatever dorky graphics you’d like from this site.