Archive

Posts Tagged ‘software architecture’

Out Of One, Many

July 6, 2012 2 comments

A Software Product Line  (SPL) can be defined as “a set of software-intensive systems that share a common, managed set of features satisfying the specific needs of a particular market segment or mission and that are developed from a common set of core assets in a prescribed way“.

The keys to developing and exploiting the value of an SPL are

  • specifying the interfaces and protocols between app components and infrastructure components
  • the granularity of the software components: 10-20K lines of code,
  • the product instantiation and test process,
  • the disciplined management of changes to the app and infrastructure components.
  • managing obsolescence of open source components/libs used in the architecture
  • keeping the requirements and design data in synch with the code base

Any others?

Highly Available Systems == Scalable Systems

June 1, 2012 4 comments

In this QCon talk: “Building Highly Available Systems In Erlang“, Erlang programming language creator and highly-entertaining speaker Joe Armstrong asserts that if you build a highly available software system, then scalability comes along for “free” with that system. Say what? At first, I wanted to ask Joe what he was smoking, but after reflecting on his assertion and his supporting evidence, I think he’s right.

In his inimitable presentation, Joe postulates that there are 6 properties of Highly Available (HA) systems:

  1. Isolation (of modules from each other – a module crash can’t crash other modules).
  2. Concurrency (need at least two computers in the system so that when one crashes, you can fix it while the redundant one keeps on truckin’).
  3. Failure Detection (in order to fix a failure at its point of origin, you gotta be able to detect it first)
  4. Fault Identification (need post-failure info that allows you to zero-in on the cause and fix it quickly)
  5. Live Code Upgrade (for zero downtime, need to be able to hot-swap in code for either evolution or bug fixes)
  6. Stable Storage (multiple copies of data; distribution to avoid a single point of failure)

By design, all 6 HA rules are directly supported within the Erlang language. No, not in external libraries/frameworks/toolkits, but in the language itself:

  1. Isolation: Erlang processes are isolated from each other by the VM (Virtual Machine); one process cannot damage another and processes have no shared memory (look, no locks mom!).
  2. Concurrency: All spawned Erlang processes run in parallel – either virtually on one CPU, or really, on multiple cores and processor nodes.
  3. Failure Detection: Erlang processes can tell the VM that it wants to detect failures in those processes it spawns. When a parent process spawns a child process, in one line of code it can “link” to the child and be auto-notified by the VM of a crash.
  4. Fault Identification: In Erlang (out of band) error signals containing error descriptors are propagated to linked parent processes during runtime.
  5. Live Code Upgrade: Erlang application code can be modified in real-time as it runs (no, it’s not magic!)
  6. Stable Storage: Erlang contains a highly configurable, comically named database called “mnesia” .

The punch line is that systems composed of entities that are isolated (property 1) and concurrently runnable (property 2) are automatically scalable. Agree?

Starting Point

Unless you’re an extremely lucky programmer or you work in academia, you’ll spend most of your career maintaining pre-existing, revenue generating software for your company.  If your company has multiple products and you don’t choose to stay with one for your whole career, then you’ll also be hopping from one product to another (which is a good thing for broadening your perspective and experience base – so do it).

For your viewing pleasure, I’ve sketched out the structure and provided some code-level metrics on the software-intensive product that I recently started working on. Of course, certain portions of the graphic have been redacted for what I hope are obvious reasons.

How about you? What does your latest project look like?

Visualizing And Reasoning About

April 26, 2012 2 comments

I recently read an interview with Grady Booch in which the interviewer asked him what his proudest technical achievement was. Grady stated that it was his involvement in the creation of the Unified Modeling Language (UML). Mr. Booch said it allowed for a standardized way (vs. ad-hoc) of “visualizing and reasoning about software” before, during, and/or after its development.

Product Line Blueprint

April 2, 2012 2 comments

Here it is, the blueprint (patent pending) you’ve been waiting for:

Need a little less abstraction? Well, how about this refinement:

Piece of cake, no? It’s easy to “figure out“:

  • the number of layers needed in the platform,
  • the functionality and connectivity within and between each of the layers in the stack
  • the granularity of the peer entities that go into each layer and which separates the layers
  • the peer-to-peer communication protocols and dependencies within each layer
  • the interfaces provided by, and required by, each layer in the stack
  • what your horizontally, integrate-able App component set should be for specific product instantiations
  • how much time, how much money, and how many people it will take to stand up the stack
  • how many different revenue-generating product variants will initially be needed for economic viability
  • how to secure all the approvals needed
  • how to manage the inevitable decrease in conceptual integrity and increase in entropy of the product factory stack over time – the maintenance problem

Perhaps easiest of all is the last bullet; the continuous, real-time management of the core asset base IF the product factory stack is actually built and placed into operation. After all, it’s not like trying to herd cats, right?

Please feel free to use this open source (LOL!) product line template to instantiate, build, and exploit your own industry and domain specific product line(s). Enough intellectualizing and “strategizing” about doing it in useless committees, task forces, special councils, tiger teams, and blue ribbon panels. There’s money to be made, joy to be distributed, and toes to be stepped on; so just freakin’ do it.

Is this post still too abstract to be of any use? Let’s release some more helium from our balloon and descend from the sky just a wee bit more so that we can get a glimpse of what is below us. Try out “revision 0” of this blueprint instantiation for a hypothetical producer of radar systems:

Did you notice the increase in tyranny of detail and complexity as we transcended the 3 levels of abstraction in this post? Well, it gets worse if we continue on cuz we don’t yet have enough information, knowledge, or understanding to start cutting code, building, testing, and standing up the stack – not nearly enough. Thus, let’s just stop right here so we can retain a modicum of sanity. D’oh! Too late!

Dream, Mess, Catastrophe

March 30, 2012 3 comments

To build high quality, successful, long-lived, “Big” software, you must design it in terms of layers (that’s why the ISO ISO model for network architecture has 7, crisply defined layers). If you don’t leverage the tool of layering (and its close cousin – leveling) in an attempt to manage complexity, then: your baby won’t have much conceptual integrity; you’ll go insane; and you’ll be the unproud owner of a big ball of mud that sucks down maintenance funds like a Dyson and may crumble to pieces at the slightest provocation. D’oh!

The figure below shows a reference model for a layered application. Note that even though we have a neat stack, we can’t tell if we have a winner on our hands.

By adding the inter-layer dependencies to the reference architecture, the true character of our software system will be revealed:

In the “Maintenance Dream“, the inter-layer APIs are crisply defined and empathetically exposed in the form a well documented interfaces, abstractions, and code examples. The programmer(s) of a given layer only have to know what they have to provide to the users above them and what the next layer below lovingly provides to them. Ah, life is good.

Next, shuffle on over to the “Maintenance Mess“. Here, we have crisply defined layers, but the allocation of functionality to the layers has been hosed up ( a violation of the principle of “leveling“) and there’s a beast in the making. Thus, in order for App Layer programmers to be productive, they have to stuff their head with knowledge/understanding of all the sub-layer APIs to get their jobs done. Hopefully, their heads don’t explode and they don’t run for the exits.

Finally, skip on over to the (shhh!) “Maintenance Catastrophe“. Here, we have both a leveling mess and an incoherent set of incomprehensible (to mere mortals)  inter-layer APIs. In the worst case: the layers aren’t discernible from one another; it takes “forever” to on-board new project members; it takes forever to fix bugs; it takes forever to add features; and it takes an heroic effort to keep the abomination alive and kicking. Double D’oh!

Forever == Lots Of Cash

In orgs that have only ever created “Maintenance Messes and Catastrophies“, since they’ve never experienced a “Maintenance Dream“, they think that high maintenance costs, busted schedules, and buggy releases are the norm. How do you explain the color green to someone who’s spent his/her whole life immersed in a world of red?

World Class Help

March 20, 2012 11 comments

I’m currently transitioning from one software project to another. After two years of working on a product from the ground up, I will be adding enhancements to a legacy system for an existing customer.

The table below shows the software technologies embedded within each of the products. Note that the only common attribute in the table is C++, which, thank god, I’m very proficient at. Since ACE, CORBA, and MFC have big, complicated, “funkyAPIs with steep learning curves, it’s a good thing that “training” time is covered in the schedule as required by our people-centric process. 🙂

I’m not too thrilled or motivated at having to spin up and learn ACE and CORBA, which (IMHO) have had their 15 minutes of fame and have faded into history, but hey, all businesses require maintenance of old technologies until product replacement or retirement.

I am, however, delighted to have limited e-access to LinkedIn connection Steve Vinoski. Steve is a world class expert in CORBA know-how who’s co-authored (with Michi Henning) the most popular C++ CORBA programming book on the planet:

Even though Steve has moved on (C++ -> Erlang, CORBA -> REST), he’s been gracious enough to answer some basic beginner CORBA questions from me without requiring a consulting contract 🙂 Thanks for your generosity Steve!

Messy Mess

January 30, 2012 2 comments

A Bunch Of STIFs

January 28, 2012 4 comments

Grady Booch states that good software architectures evolve incrementally via a progressive sequence of STable Intermediat Forms (STIFs). At each point of equilibrium, the “released” STIF is exercised by an independent group of external customers and/or internal testers. Defects, unneeded functionality, missing functionality, and unmet “ilities” are ferreted out and corrected early and often.

The alternative to a series of STIFs is the ubiquitous, one-shot, Unstable Fuming Fiasco (UFF):

Note: After I converted this draft post into a publishable one and queued it up, I experienced a sense of deja-vu. As a consequence, I searched back through my draft (91) and published (987) post lists. I found this one. D’oh! and LOL! I’m sure I’ve repeated myself many times during my blogging “career“, but hey, a steady drumbeat is more effective than a single cymbal crash. No?

A Thing In Context

January 11, 2012 2 comments