Posts Tagged ‘Erlang’

Acting In C++

May 21, 2015 7 comments

Because they’re a perfect fit for mega-core processors and they’re safer and more enjoyable to program than raw, multi-threaded, systems, I’ve been a fan of Actor-based concurrent systems ever since I experimented with Erlang and Akka (via its Java API). Thus, as a C++ programmer, I was excited to discover the “C++ Actor Framework” (CAF). Like the Akka toolkit/runtime, the CAF is based on the design of the longest living, proven, production-ready, pragmatic, actor-based, language that I know of – the venerable Erlang programming language.

Curious to check out the CAF, I downloaded, compiled, and installed the framework source code with nary a problem (on CentOS 7.0, GCC 4.8.2). I then built and ran the sample “Hello World” program presented in the user manual:


As you can see, the CAF depends on the use one of my favorite C++11 features – lambda functions – to implement actor behavior.

The logic in main() uses the CAF spawn() function to create two concurrently running, message-exchanging (messages of type std::string), actors named “mirror” and “hello_world“. During the spawning of the “hello_world” actor, the main() logic passes it a handle (via const actor&) to the previously created “mirror” actor so that “hello_world” can asynchronously communicate with “mirror” via message-passing. Then, much like the parent thread in a multi-threaded program “join()“s the threads it creates to wait for the threads to complete their work, the main() thread of control uses the CAF await_all_actors_done() function to do the same for the two actors it spawns.

When I ran the program as a local C++ application in the Eclipse IDE, here is the output that appeared in the console window:

CAF mirror

The figure below visually illustrates the exchange of messages between the program’s actors during runtime. Upon being created, the “hello_world” actor sends a text message (via sync_send()) containing “Hello World” as its payload to the previously spawned “mirror” actor. The “hello_world” actor then immediately chains the then() function to the call to synchronously await for the response message it expects back from the “mirror” actor. Note that a lambda function that will receive and print the response message string to the console (via the framework’s thread-safe aout object) is passed as an argument into the then() function.

CAF HW actors

Looking at the code for the “mirror” actor, we can deduce that when the framework spawns an instance of it, the framework gives the actor a handle to itself and the actor returns the behavior (again, in the form of a lambda function) that it wants the framework to invoke each time a message is sent to the actor. The “mirror” actor behavior: prints the received message to the console, tells the framework that it wants to terminate itself (via the call to quit()), and returns a mirrored translation of the input message to the framework for passage back to the sending actor (in this case, the “hello_world” actor).

Note that the application contains no explicit code for threads, tasks, mutexes, locks, atomics, condition variables, promises, futures, or thread-safe containers to futz with. The CAF framework hides all the underlying synchronization and message passing minutiae under the abstractions it provides so that the programmer can concentrate on the application domain functionality. Great stuff!

I modified the code slightly so that the application actors exchange user-defined message types instead of std::strings.

req-resp CAF

Here is the modified code:

struct ReqMessage {
  int32_t anInt32_t;
  double aDouble;
  string aString;

struct RespMessage {
  enum class AckNack{ACK, NACK};
  AckNack response;

void printMsg(event_based_actor* self, const ReqMessage& msg) {
  aout(self) << "anInt32_t = " << msg.anInt32_t << endl
             << "aDouble = " << msg.aDouble << endl
             << "aString = " << msg.aString << endl
             << endl;

void fillReqMessage(ReqMessage& msg) {
  msg.aDouble = 55.5;
  msg.anInt32_t = 22;
  msg.aString = "Bulldozer00 Rocks!";

behavior responder(event_based_actor* self) {
  // return the (initial) actor behavior
  return {
  // a handler for messages of type ReqMessage
  // and replies with a RespMessage
  [=](const ReqMessage& msg) -> RespMessage {
    // prints the content of ReqMessage via aout
    // (thread-safe cout wrapper)
    printMsg(self, msg);
    // terminates this actor *after* the return statement
    // ('become' a quitter, otherwise loops forever)
    // reply with an "ACK"
    return RespMessage{RespMessage::AckNack::ACK};

void requester(event_based_actor* self, const actor& responder) {
  // create & send a ReqMessage to our buddy ...
  ReqMessage req{};
  self->sync_send(responder, req).then(
  // ... wait for a response ...
  [=](const RespMessage& resp) {
    // ... and print it
    RespMessage::AckNack payload{resp.response};
    string txt =
      (payload == RespMessage::AckNack::ACK) ? "ACK" : "NACK";
    aout(self) << txt << endl;

int main() {
  // create a new actor that calls 'responder()'
  auto responder_actor = spawn(responder);
  // create another actor that calls 'requester(responder_actor)';
  spawn(requester, responder_actor);
  // wait until all other actors we have spawned are done
  // run cleanup code before exiting main

And here is the Eclipse console output:
req-resp CAF output

Categories: C++, C++11 Tags: , , ,

Scaling Up And Out

January 8, 2015 2 comments

Ever since I discovered the venerable Erlang programming language and the great cast of characters (especially Joe Armstrong) behind its creation, I’ve been chomping at the bit to use it on a real, rent-paying, project. That hasn’t happened, but the next best thing has. I’m currently in the beginning stage of a project that will be using “Akka” to implement a web-based radar system gateway.

Akka Logo

Akka is an Erlang-inspired toolkit and runtime system for the JVM that elegantly supports the design and development of highly concurrent and distributed applications. It’s written in Scala, but it has both Scala and Java language bindings.

Akka Def

Jonas Boner, the creator of Akka, was blown away by the beauty of Erlang and its ability to support “nine nines” of availability – as realized in a commercial Ericsson telecom product. At first, he tried to get clients and customers to start using Erlang. But because Erlang has alien, Prologue-based syntax, and its own special VM/development infrastructure, it was a tough sell. Thus, he became determined to bring”Erlangness” to the JVM to lower the barriers to entry. Jonas seems to be succeeding in his crusade.

At the simplest level of description, an Akka-based application is comprised of actors running under the control of an overarching Akka runtime system container. An actor is an autonomous, reactive entity that contains state, behavior, and an input mailbox from which it consumes asynchronously arriving messages. An actor communicates with its parents/siblings by issuing immutable, fire-and-forget (or asynchronous request-and-remember), messages to them. The only way to get inside the “head” of an actor is to send messages to it. There are no backdoors that allow other actors to muck with the inner state of an actor during runtime. Like people, actors can be influenced, but not explicitly coerced or corrupted, by others. However, also like people, actors can be terminated for errant behavior by their supervisors. 🙂

For applications that absolutely require some sort of shared state between actors, Akka provides transactors (and agents). A transactor is an implementation of software transactional memory that serializes access to shared memory without the programmer having to deal with tedious, error-prone, mutexes and locks. In essence, the Akka runtime unshackles programmers from manually handling the frustrating minutiae (deadlocks, race conditions (Heisenbugs), memory corruption, scalability bottlenecks, resource contention) associated with concurrent programming. Akka allows programmers to work at a much more pleasant, higher level of abstraction than the traditional, bare bones, threads/locks model. You spend more time solving domain-level problems than battling with the language/library implementation constraints.

Akka System

Actor Transactor

You design an Akka based system by, dare I say, exercising some upfront design brain muscle:

  • decomposing and allocating application functionality to actors,
  • allocating data to inter-actor messages and transactors,
  • wiring the actors together to structure your concurrent system as a network of cooperating automatons.

The Akka runtime supervises the application by operating unobtrusively in the background:

  • mapping actors to OS threads and hardware cores,
  • activating an actor for each message that appears in its mailbox,
  • handling actor failures,
  • realizing the message passing mechanisms.

Gee, I wonder if the agile TDD and the “no upfront anything” practices mix well with concurrent, distributed system design. Silly me. Of course they do. Like all things agile, they’re universally applicable in all contexts. (Damn, I thought I’d make it through this whole post without dissing the dogmatic agile priesthood, but I failed yet again.)

Akka concurrent example

Akka not only makes it easy to scale up on a multi-core processor, Akka makes it simple to scale out and distribute (if needed) your concurrent design amongst multiple computing nodes via a URL-based addressing scheme.

Akka distributed example

There is much more deliciousness to Akka than I can gush about in this post (most notably, actor supervision trees, futures, routers, message dispatchers, etc), but you can dive deeper into Akka (or Erlang) if I’ve raised the hairs on the back of your geeky little neck. The online Akka API documentation is superb. I also have access to the following three Akka books via my safaribooksonline account:

Akka Books

Since C++ is my favorite programming language, I pray that the ISO C++ committee adds Erlang-Akka like abstractions and libraries to the standard soon. I know that proposals for software transactional memory and improved support for asynchronous tasking and futures are being worked on as we speak, but I don’t know if they’ll be polished enough to add to the 2017 standard. Hurry up guys :).

Cpp Akka

Categories: C++, technical Tags: , ,

Regime Change

November 12, 2014 4 comments

Revolution is glamorous and jolting; evolution is mundane and plodding. Nevertheless, evolution is sticky and long-lived whereas revolution is slippery and fleeting.

As the figure below from Neal Ford’s OSCON “Functional Thinking” talk reveals, it took a glacial 16 years for Object-Oriented Programming (OOP) to firmly supplant Procedural Programming (PP) as the mainstream programming style of choice. There was no revolution.

Starting with, arguably, the first OOP language, Simula, the subsequent appearance of Smalltalk nudged the acceptance of OOP forward. The inclusion of object-oriented features in C++ further accelerated the adoption of OOP. Finally, the emergence of Java in the late 90’s firmly dislodged PP from the throne in an evolutionary change of regime.

I suspect that the main reason behind the dethroning of PP was the ability of OOP to more gracefully accommodate the increasing complexity of software systems through the partitioning and encapsulation of state.

Proc to OO

Mr. Ford asserts, and I tend to agree with him, that functional programming is on track to inherit the throne and relegate OOP to the bench – right next to PP. The main force responsible for the ascent of FP is the proliferation of multicore processors. PP scatters state, OOP encapsulates state, and FP eschews state. Thus, the FP approach maps more naturally onto independently running cores – minimizing the need for performance-killing synchronization points where one or more cores wait for a peer core to finish accessing shared memory.

regime change

The mainstream-ization of FP can easily be seen by the inclusion of functional features into C++ (lambdas, tasks, futures) and the former bastion of pure OOP, Java (parallel streams). Rather than cede the throne to pure functional languages like the venerable Erlang, these older heavyweights are joining the community of the future. Bow to the king, long live the king.

Categories: C++ Tags: , , ,

Parallelism And Concurrency

In the beginning of Robert Virding’s brilliant InfoQ talk on Erlang, he distinguishes between parallelism and concurrency. Parallelism is “physical“, having to do with the static  number of cores and processors in a system. Concurrency is “abstract“, having to do with the number of dynamic application processes and threads running in the system. To relate the physical with the abstract, I felt compelled to draw this physical-multi-core, physical-multi-node, abstract-multi-process, abstract-multi-thread diagram:


It’s not much different than the pic in this four year old post: PTCPN. It’s simply a less detailed, alternative point of view.

Which Path?

April 26, 2013 7 comments

Please peruse the graphic below and then answer these questions for BD00: Is it a forgone conclusion that object-oriented development is the higher quality path from requirements to source code for all software-intensive applications? Has object-oriented development transitioned over time from a heresy into a dogma?

SA vs OO

With the rising demand for more scaleable, distributed, fault-tolerant, concurrent systems and the continuing maturation of functional languages (founded on immutability, statelessness, message-passing (e.g. Erlang, Scala)), choosing between object-oriented and function-oriented technical approaches may be more important for success than choosing between agile development methodologies.

So Professional That It’s Unprofessional

November 3, 2012 4 comments

Mike Williams is one of the big three Erlang creators (along with Joe Armstrong and Robert Virding) and the developer of the first Erlang Virtual Machine. Since then, he’s “moved up” and has been working as a software engineering manager for 20+ years. In his InfoQ talk titled “The Ideal Programmer – Why They Don’t Exist and How to Manage Without Them?“, Mike presents this hilarious slide:

It’s hilarious because, if you browse web sites like and, you’ll find tons of similar, impossible-to-satisfy job descriptions. Everybody, especially the job description writer, knows that exhaustive “requirements” lists like these are a crock of BS. This practice is so professional that it’s unprofessional. So, why does it persist?

Great, But Weird And Slow

August 31, 2012 Leave a comment

Damien Katz is the soft-spoken Couchbase CTO and creator of the Apache CouchDB (written in Erlang and C/C++). In this InfoQ presentation, “Erlang, the Language from the Future?“, Damien praises the Erlang programming language. But he also points out its turds: weirdness and slowness.

Damien says that Erlang’s weird Prolog-like syntax prevents massive adoption; which prevents lack of massive investment; which prevents massive VM performance improvement.

To prove his point, Mr. Katz uses Java as an example of “getting it right“. When Java arrived on the scene in the mid-90’s, it was dirt slow compared to C and C++. But because Java’s syntax was C-like and it didn’t suffer from the frustrating quirks and gotchas of C/C++, it was embraced quickly and massively by the software industry. This lead to massive investment, which lead to large JVM performance increases and a boatload of productivity-enhancing development tools.

I like Erlang a lot for the concurrent, distributed type of software I write. However, Damian hits the nail right on the head. What do you think?

Best Actor Award

I recently watched (Trifork CTO and Erjang developer) Kresten Krab Thorup give this terrific talk: “Erlang, The Road Movie“.  In his presentation, Kresten suggested that the 20+ year reign of the “objectsprogramming paradigm is sloooowly yielding to the next big problem-solving paradigm: autonomous “actors“. Using Thomas Kuhn‘s well known paradigm-change framework, he presented this slide (which was slightly augmented by BD00):

Kresten opined that the internet catapulted Java to the top of the server-side programming world in the 90s. However, the new problems posed by multi-core, cloud computing, and the increasing need for scalability and fault-tolerance will displace OOP/Java with actor-based languages like Erlang. Erlang has the upper-hand because it’s been evolved and battle-tested for over 20 years. It’s patiently waiting in the wings.

The slide below implies that the methods of OOP-based languages designed to handle post-2000 concurrency and scalability problems are rickety graft-ons; whereas the features and behaviors required to wrestle them into submission are seamlessly baked-in to Erlang’s core and/or its OTP library.

So, what do you think? Is Mr. Thorup’s vision for the future direction of programming correct? Is the paradigm shift underway? If not, what will displace the “object” mindset in the future. Surely, something will, no?

Too much of my Java programs are boilerplate code. – Kresten Krab Thorup

Too much of my C++ code is boilerplate code. – Bulldozer00

Java either steps up, or something else will. – Cameron Purdy

Highly Available Systems == Scalable Systems

June 1, 2012 4 comments

In this QCon talk: “Building Highly Available Systems In Erlang“, Erlang programming language creator and highly-entertaining speaker Joe Armstrong asserts that if you build a highly available software system, then scalability comes along for “free” with that system. Say what? At first, I wanted to ask Joe what he was smoking, but after reflecting on his assertion and his supporting evidence, I think he’s right.

In his inimitable presentation, Joe postulates that there are 6 properties of Highly Available (HA) systems:

  1. Isolation (of modules from each other – a module crash can’t crash other modules).
  2. Concurrency (need at least two computers in the system so that when one crashes, you can fix it while the redundant one keeps on truckin’).
  3. Failure Detection (in order to fix a failure at its point of origin, you gotta be able to detect it first)
  4. Fault Identification (need post-failure info that allows you to zero-in on the cause and fix it quickly)
  5. Live Code Upgrade (for zero downtime, need to be able to hot-swap in code for either evolution or bug fixes)
  6. Stable Storage (multiple copies of data; distribution to avoid a single point of failure)

By design, all 6 HA rules are directly supported within the Erlang language. No, not in external libraries/frameworks/toolkits, but in the language itself:

  1. Isolation: Erlang processes are isolated from each other by the VM (Virtual Machine); one process cannot damage another and processes have no shared memory (look, no locks mom!).
  2. Concurrency: All spawned Erlang processes run in parallel – either virtually on one CPU, or really, on multiple cores and processor nodes.
  3. Failure Detection: Erlang processes can tell the VM that it wants to detect failures in those processes it spawns. When a parent process spawns a child process, in one line of code it can “link” to the child and be auto-notified by the VM of a crash.
  4. Fault Identification: In Erlang (out of band) error signals containing error descriptors are propagated to linked parent processes during runtime.
  5. Live Code Upgrade: Erlang application code can be modified in real-time as it runs (no, it’s not magic!)
  6. Stable Storage: Erlang contains a highly configurable, comically named database called “mnesia” .

The punch line is that systems composed of entities that are isolated (property 1) and concurrently runnable (property 2) are automatically scalable. Agree?

The Expense Of Defense

November 19, 2011 3 comments

The following “borrowed” snippet from a recent Steve Vinoski QCon talk describes the well worn technique of defensive programming:

Steve is right, no? He goes on to deftly point out the expenses of defensive programming:

Steve is right on the money again, no?

Erlang and the (utterly misnamed)  Open Telecom Platform (OTP) were designed to obviate the need for the defensive programming “idiom” style of error detection and handling. The following Erlang/OTP features force programmers to address the unglamorous error detection/handling aspect of a design up front, instead of at the tail end of the project where it’s often considered a nuisance and a second class citizen:

Even in applications where error detection and handling is taken seriously upfront (safety-critical and high availability apps), the time to architect, design, code, and test the equivalent “homegrown” capabilities can equal or exceed the time to develop the app’s “fun” functionality. That is, unless you use Erlang.

%d bloggers like this: