Bulldozer00's Blog

When creating an architecture or detailed design of a system component, up to three forms of design evaluation can be performed:

• A: Continuous evaluation by the designer during the process
• B: Evaluation by a peer group at one or more review events during the process
• C: Evaluation by an expert outsider group after A and B

These three forms of evaluation are illustrated in the sequence of diagrams below. Note that as each evaluation type is progressively added to the mix, a new feedback loop is introduced into the system.

During A, the conceive/evaluate/repair loop is performed at the speed of thought within the isolated mind of the designer. As the designer applies his skills to his understanding of the problem to be solved, alternative component structures and configurations can pop up instantaneously during each of many high frequency loop traversals. At some point, the designer concludes that his design solution is stable enough for external scrutiny and/or ready for the next step of realization.

During B, the designer walks through his creation in front of a group of peers. He explains the structure and behavior mechanisms of his design and how it solves the problem. If the peer group is qualified, prepared, and objective (QPO), at least a portion of the feedback the group produces will be valuable to the designer and he will use it to improve his design. However, unless all three QPO pre-conditions are satisfied, the peer review process will be a huge waste of time and money.

Jetting in an expert outsider group is the riskiest type of evaluation. Assuming that the QPO criteria is satisfied, the fact that the expert outsider group has no real “skin in the game” after it vacates the premises should be a cause for concern to anyone contemplating the use of the technique.

In general, the more negative feedback loops incorporated into a process, the more likely the process is to produce its intended output. As the final figure above implies, incorporating all three types of evaluation into a design review process can lead to a high quality design. However, blindly ignoring the QPO criteria and/or failing to address the “skin in the game” risk can increase cost, lengthen schedule, and lay waste to your well-intentioned review process – without you ever knowing that it occurred. As ever, the devil is in the details.