Archive
DataLoggerThread
The figure below models a program in which a pipeline of worker threads communicate with each other via message passing. The accordion thingies ‘tween the threads are message queues that keep the threads loosely coupled and prevent message bursts from overwhelming downstream threads.
During the process of writing one of these multi-threaded programs to handle bursty, high rate, message streams, I needed a way to periodically extract state information from each thread so that I could “see” and evaluate what the hell was happening inside the system during runtime. Thus, I wrote a generic “Data Logger” thread and added periodic state reporting functionality to each worker thread to round out the system:
Because the reporting frequency is low (it’s configurable for each worker thread and the default value is once every 5 seconds) and the state report messages are small, I didn’t feel the need to provide a queue per worker thread – YAGNI.
The figure below shows a more detailed design model of the data logging facility in the form of a “bent” UML class diagram. Upon construction, each DataLoggerThread object can be configured to output state messages to a user named disk file and/or the global console during runtime. The rate at which a DataLoggerThread object “pops” state report messages from its input queue is also configurable.
The DataLoggerThread class provides two different methods of access to user code at runtime:
void DataLoggerThread::record_txt_block(const Data&)
and
void DataLoggerThread::operator<<(const Data&).
Objects of the DataLoggerThread class run in their own thread of execution – transparently in the background to mainline user code. On construction, each object instance creates a mutex-protected, inter-thread queue and auto-starts its own thread of operation behind the scenes. On destruction, the object gracefully self-terminates. During runtime, each DataLoggerThread object polls its input queue and formats/writes the queue entries to the global console (which is protected from simultaneous, multiple thread access by a previously developed CoutMonitor class) and/or to a user-named disk log file. The queue is drained of all entries on each (configurable,) periodic activation by the underlying (Boost) threads library.
DataLoggerThread objects pre-pend a “milliseconds since midnight” timestamp to each log entry just prior to being pushed onto the queue and a date-time stamp is pre-pended to each user supplied filespec so that file access collisions don’t occur between multiple instances of the class.
That’s all I’m gonna disclose for now, but that’s OK because every programmer who writes soft, real-time, multi-threaded code has their own homegrown contraption, no?
Renunciation
I was listening to an Eckhart Tolle talk the other day (via an mp3 file deposited on a USB stick plugged into my new Subaru WRX audio system). He started talking about the key to inner peace. In order to prepare his audience for the bombshell he was about to launch, he asked them to “please don’t be shocked“. Eckhart then said, in his characteristically slow, deliberate, and relaxed tone, that the key to awakening and inner peace is….. the renunciation of thought. D’oh, and WTF?
Years ago, when I was more lost than I am today, I would have thought that Mr. Tolle was a charlatan on the level of Jimmy Swaggart and Jim Bakker. Today, knowing what I know, but can’t express in words, I think he is wise. How about you? What do you think?
Note: I usually use Microsft Visio to generate the dorky diagrams on this blog. The pic in this post is my first one using the freely downloadable Inkscape package.
One Size Fits All
On Bjarne Stroustrup’s FAQ page, he state’s his opinion on the utility of object oriented programming:
The strength of OOP is that there are many problems that can be usefully expressed using class hierarchies – the main weakness of OOP is that too many people try to force too many problems into a hierarchical mould. Not every program should be object-oriented. As alternatives, consider plain classes, generic programming, and free-standing functions (as in math, C, and Fortran).
The same could be said of organizations of people, no? Of course, in a purposeful org of people and machines, the goal is to produce and distribute material wealth to all stakeholders: products/services to its customers and financial well-being to its employees and owners. Thus, some sort of controller-controllee structure is required to keep the org from deviating too far from its purpose. There are alternative structures to hierarchy but, as Bjarne sez, “too many people try to force too many problems into a hierarchical mould“. Bummer.
A Missing Core Value?
I’d venture to say that every technology company has phrases similar to “elegant products“, “technical excellence“, “innovative solutions“, and “quality first” smartly written in its mission and/or core values statements. I’d also venture to say that “schedule is king” is not explicitly inscribed in those WORN documents.
Regardless of what is espoused in their cutesy mission and core values statements, all mediocre and underperforming corpricracies operate day-to-day as if “schedule is king” is their top priority. How many times have you heard managers say the words “quality“, “elegance“, or “excellence” when discussing or reviewing a project? Now, how many times have you heard the word “schedule” uttered by managers?
If “quality“, “elegance“, or “excellence” are never mentioned because they’re “auto-assumed” to be present in all project endeavors, then why write them down? If “meeting schedule at all costs” is really what drives day-to-day behavior in the DYSCO, then why not write it down and put it at the top of the list?
Cool Text
Cool Text is a FREE graphics generator for web pages and anywhere else you need an impressive logo without a lot of design work. Simply choose what kind of image you would like. Then fill out a form and you’ll have your own custom image created on the fly.
Checkout what you can do at cooltext.com in the blink of an eye:
Learn To Estimate?
Item number 50 in “97 Things Every Programmer Should Know” is titled:
Since most schedules magically appear from the heavens without any input from below, why is there a need to learn how to estimate? If, by chance, schedule inputs ARE solicited from those who will do the work, they’re often ignored since heavenly commitments have already been made behind the scenes.
Zero Revenue, Zero Cost
Since my cafepress.com T-shirt shop, “The Frontal Assault Idiot“, opened to rave reviews earlier this month, the dough has been rolling in. Check out the orders report below (especially the notice in red) for the fiscal month of July 2011. LOL!
I guess I’ll have to bring in some hot shot consultants to help me get my expenses down and fend off my creditors. But wait! There are no expenses. I can stay in business. Whoo Hoo!
Botted?
I’ve been blogging for over 2 years now. Until a few days ago, the maximum number of hits that this blog had absorbed in one day was 149. On 07/29/11, my blogs stats chart zoomed to 291 hits:
After zeroing in on the July 29 date, I discovered that 231 of the hits came from Google:
From looking at the content of the referral links, I’ve concluded that a google bot had crawled the site and extracted/stored a bunch of the dorky images that I post daily. Do you think that’s what happened?
Childlike Humility
This quote from Amazon Kindle: Most Highlighted Passages of All Time really socked me in the jaw. Does it do anything for you?
“As-Built” Vs “Build-To”
The figure below shows one (way too common) approach for developing computer programs. From an “understanding” in your head, you just dive into the coding state and stay there until the program is “done“. When it finally is “done“: 1) you load the code into a reverse engineering tool, 2) press a button, and voila, 3) your program “As-Built” documentation is generated.
For trivial programs where you can hold the entire design in your head, this technique can be efficient and it can work quite well. However, for non-trivial programs, it can easily lead to unmaintainable BBoMs. The problem is that the design is “buried” in the code until after the fact – when it is finally exposed for scrutiny via the auto-generated “as-built” documentation.
With a dumb-ass reverse engineering tool that doesn’t “understand” context or what the pain points in a design are, the auto-generated documentation is often overly detailed, unintelligible, camouflage in which a reviewer/maintainer can’t see the forest for the trees. But hey, you can happily tick off the documentation item on your process checklist.
Two alternative, paygo types of development approaches are shown below. During development, the “build-to” design documentation and the code are cohesively produced manually. Only the important design constructs are recorded so that they aren’t buried in a mass of detail and they can be scrutinized/reviewed/error-corrected in real-time during development – not just after the fact.
I find that I learn more from the act of doing the documentation than from pushing an “auto-generate” button after the fact. During the effort, the documentation often speaks to me – “there’s something wrong with the design here, fix me“.
Design is an intimate act of communication between the creator and the created – Unknown
Of course, for developers, especially one dimensional extreme agilista types who have no desire to “do documentation” or learn UML, the emergence of reverse engineering tools has been a Godsend. Bummer for the programmer, the org he/she works for, the customer, and the code.
Why am I here?
WTF?
Meh!
D'oh!
My BTC Address
Bulldozer00's Blog 