Pragmatic Automation

Using the Firecracker with Macs

Jim Weirich wrote in to say that he has it on good authority that the KeySpan USA-19HS will properly drive a Firecracker. The Keyspan adapter comes with Mac OS X drivers that speak the handshake signals the Firecracker listens for. Huzzah! That means you can remotely control your lava lamps from your Mac.

Enjoy!

Bubble, Bubble, Build's In Trouble

Your software is being automatically built and tested on a schedule. It even sends you an email when the code doesn’t compile or pass its tests. You’re certainly ahead of most projects, but email is just so 90s. Even if you could manage to find those build failure emails amidst all that spam, you’re reading yesterday’s news. Indeed, you may already be ignoring the status of the scheduled build.

The Monitoring chapter of the book offers alternative, in-your-face, worth-getting-up-for-in-the-morning techniques for monitoring scheduled builds. The most popular technique came by way of a story contributed by Alberto Savoia. He describes how his project uses red and green lava lamps to radiate the status of their scheduled build. Better yet, those lamps are controlled using X10 devices such as those used to turn on your household lamps so that you don’t arrive home to a dark house.

Well, as you might imagine, I could hardly wait to build my very own build-monitoring lava lamp kit. And as bonus material for readers of the book, I’ve crafted a bit o’ software that integrates with CruiseControl. So now you too can enjoy red and green bubbles on your project!

Bill of Materials

To get started, you need some automation gear. Think of these gadgets as this year’s essential project accessories:

4-Piece Firecracker Automation System
This kit includes:
- 1 Firecracker Computer Interface
- 1 Transceiver Module
- 1 Lamp Module
- 1 Palm Pad Remote Control
Cost: $39.99

(Props go to the folks at x10.com for supporting this project by supplying me with a complimentary kit. It all fits in a wee box, so I can carry it from project to project.)

With that kit, you can control two lava lamps — one plugged into the transceiver module and the other plugged into the lamp module. You can optionally purchase another appliance module if you want to control two appliances. For example, you might want your build process to turn on a coffee pot when the build fails and then kick start your margarita machine when the build is fixed.
2 lamps, preferably the kind that boil red and green lava
I used the Hot Rock Lite F/X (yellow earth/blue liquid and red earth/purple liquid). Note for legal purposes that these lamps (shown in pictures below) are not LAVA(R) brand motion lamps, but those will work just as well.

Cost: $9.99 each at Target or Walmart
Pragmatic Automation X10 software
It’s an open source Java library that includes the CruiseControl plug-in, an API to make your wildest X10 dreams come true, detailed instructions, and an ever-so-useful collection of tests.

Way down deep, the library uses the Java Communications API to send bits out over the serial port and into the Firecracker Computer Interface. (Linux users will need the RXTX implementation). Michel Dalal’s Java X10 CM17A API library, an implementation of the FireCracker (CM17A) Communications Specification, is used to send out the correct 1s and 0s in response to human-friendly commands. Many thanks to him for doing all the low-level bit twiddling and sharing the goodies with us!

Cost: Free to readers of Pragmatic Project Automation

Assembling the Kit

With that hardware in hand, you’re ready to start the assembly process. The Firecracker Automation System includes instructions written for your average home electronics consumer, so your average computer/network geek should have no trouble. I’ll spare you all the gory details and instead run through a quick visual tutorial of my setup.

Start by plugging the Firecracker Computer Interface into a serial port of your scheduled build machine:

This little gem sends a wireless signal from the computer to the transceiver module. Notice that you don’t lose the serial port. You can plug another serial device into the back of Firecracker Computer Interface.

Next, plug the transceiver module into any wall outlet within your electrical wiring system. Then turn on the lamp you want the build process to light up when the build fails (the red lamp) and plug it into the transceiver module:

See that antennae on the transceiver? The transceiver picks up the RF signal sent by the Firecracker Computer Interface connected to the computer, converts it into an X10 signal, and broadcasts the X10 signal across the electrical wiring system.

Every X10 module is uniquely identified by a house code (A-P) and a unit code (1-16). By default, the transceiver is configured to listen on "A1". So when the Firecracker Computer Interface sends a signal that tells module "A1" to turn on, the device that’s connected to the transceiver—the red lamp—is turned on.

Next, plug the lamp module into a wall outlet and set its house and unit code to "A2". Then turn on the lamp you want the build process to light up when the build passes (the green lamp) and plug it into the lamp module, like so:

When the Firecracker Computer Interface sends a signal instructing the "A2" module to turn on, the transceiver picks up the signal and broadcasts it out over the electrical wires. The lamp module hears the signal and turns on the green lamp.

That’s it for assembly!

At this point it’s a good idea to make sure you can turn these lamps on and off at will using the free software (separate download) that emulates the Palm Pad Remote Control. It sends signals through the Firecracker Computer Interface, so it’s a good sanity check that you have everything hooked up correctly.

Installing the Software

OK, now you want to hook the lava lamps up to your scheduled build process running in CruiseControl. That’s where the Pragmatic Automation X10 software comes in.

The README file describes how to install and test the software in detail. The final step is to register the CruiseControl plug-in that effectively wires up the lamps to indicate the status of each CruiseControl build cycle. Just to demonstrate how easy that is, here’s the XML you need to add to CruiseControl’s config.xml file:

  <plugin name="x10publisher" classname="com.pragauto.X10Publisher"/>

  <publishers>
    <x10publisher port="COM1" passDeviceCode="A2" failDeviceCode="A1" />
  </publishers>

Edit the attributes of the <x10publisher> element as necessary for your serial port and device codes. A complete config.xml file is included in the project as a reference.

Bubbles While You Work

Once you’ve fired up CruiseControl and a build succeeds, you’ll see something like this on the console:

  BUILD SUCCESSFUL
  Total time: 10 seconds
  [cc]Jul-08 16:56:53 Project       - Project dms:  merging accumulated log files
  [cc]Jul-08 16:56:53 Project       - Project dms:  publishing build results
  [cc]Jul-08 16:56:53 X10Publisher  - Turning pass device (A2) on; fail device (A1) off ...
  [cc]Jul-08 16:56:56 Project       - Project dms:  idle
  [cc]Jul-08 16:56:56 Project       - Project dms:  next build in 1 minutes

At that point, the green lava lamp should turn on. Bask in that pleasant glow for a moment. When the lamp gets warmed up, you’ll get entertained by happy, green bubbles:

And then at some point somebody might check in code that causes the build to fail. (Hey, it happens to even the best programmers once in a while.) Here’s the early warning sign that the scheduled build is in trouble:

Eek! Notice that the ambient light in the red lamp goes on immediately. It will take a while for the red lava to heat up and start to boil. In the meantime, there’s work to be done. When you check the CruiseControl log, you’ll see something like this:

  BUILD FAILED
  file:C:/work/dms/builds/checkout/dms/build.xml:77: Tests failed! Check test reports.

  Total time: 5 seconds
  [cc]Jul-08 16:58:20 Project       - Project dms:  merging accumulated log files
  [cc]Jul-08 16:58:20 Project       - Project dms:  publishing build results
  [cc]Jul-08 16:58:20 X10Publisher  - Turning pass device (A2) off; fail device (A1) on...
  [cc]Jul-08 16:58:23 Project       - Project dms:  idle
  [cc]Jul-08 16:58:23 Project       - Project dms:  next build in 1 minutes

If it takes too long to fix the build, the red lamp will start to get angry:

The object of the game is to keep the green lamp glowing. :-)

Hearing the Build Break

You may have noticed that when the transceiver module turns on and off it makes a fairly loud snapping sound. That sound is caused by the mechanical relay inside. (The lamp module doesn’t make that sound, I suspect because it’s a low-current device that doesn’t use a mechanical relay.)

This audible feedback turns out to be quite handy if your team is working in close quarters. The "snap" could be yet another sound in your project soundscape. By plugging the red lamp into the transceiver module (A1), you’ll hear that snapping sound just before the red lamp turns on. So if you’re heads down (without headphones), the sound will alert you to a build failure. Indeed, you’ll hear the build break!

Important Safety Notes

These lamps get insanely hot. If you don’t let them cool down for at least five minutes before touching the glass, you’ll find out just how hot. (Yes, even the green one is hot.)

The instructions for my lamps note that they perform best if operated for less than ten hours at a time. Your local fire marshal would certainly agree. But who will remember to turn off the lamps at the end of every programming day? Well, that’s a job for the ol’ computer. Just write a shutdown program that uses the Pragmatic Automation X10 software to turn off the lamps. For example:

  new X10Device("COM1", "A1").off();
  new X10Device("COM1", "A2").off();

Then schedule an at or cron job on the scheduled build machine that runs the shutdown program at some hour of every evening, and another program that turns the lamps on every morning.

What About My Macintosh?

The Firecracker Computer Interface plugs into a serial port. Modern day Macintosh computers don’t have a serial port. Sadly, none of the USB-to-serial converters I tried worked with the Firecracker Computer Interface. (I’d love to hear otherwise.)

You can purchase X10 kits that plug into USB, but I chose to use the Firecracker kit because it’s relatively cheap and it’s what is described in the book. Thankfully, by writing and testing most of this software against a mock device, I was able to do the majority of the development on my PowerBook. :-)

Conclusion

It’s relatively easy and inexpensive to make build monitoring a spectator sport. Put the lava lamps in a highly-visible area of your project and use them as visual (and audible) feedback devices, and to show off your programming prowess, of course. And that’s just the beginning. You can use any appliance to monitor anything that’s valuable to your team.

Are you using feedback devices to spice up your project? If so, please share your story.

Enjoy!

Screeching Animatronic Monkeys

Suppose you’ve just launched a new web application. Being a savvy automator, you decide to set up monitors for significant system events like, say, new users signing up. Now it’s always good to pick a physical device that gives visual or audible feedback that in some way matches the logical event you’re monitoring. So what kind of feedback goes along with a new user signing up?

Ian Bishrey couldn’t exactly wire up a human, so he went with the next best thing: a screeching Wowwee Alive! Monkey.

You know you want one of these on your project, and Ian has instructions and code for how he hacked the remote controller and wrote a Java-based serial interface (with tests) to send commands to the animatronic monkeys.

In the email he sent, Ian concluded by stating matter of factly "Using the remote means we can control an army of chimps simultaneously." I had to read that sentence a couple times, and each time the mental images got progressively more frightening. Added to which, Ian mentioned in passing that they also use the monkeys to give feedback on "low NLP linguistic classification confidence score events". So to summarize, we have a web 2.0 application that’s taking on new users and in some way processing natural languages, all being monitored by a potential army of monkeys.

You know, when I challenged readers of the automation book to find creative ways to monitor their software projects, I never imagined it could come to this…

Chumby

Chumby looks to be a fun way to monitor automation in and around projects. It’s a portable Internet-connected device that displays visual widgets. Just "trust the Chumby" and it autodiscovers your wifi network.

But what makes Chumby special is the open and hackable design. Want a custom widget? Make one. Got some hardware you’d like to integrate? Plug it into the chumbilical.

This is going to be fun to watch. Please send pictures of Chumby at work on your software project!

Adding an RSS Feed to Subversion

Bill Bumgarner expertly steps you through how to hook an RSS feed into a local Subversion server.

More CVS Monitoring

Ethan wrote in to give us a heads-up on his new-and-improved CVS logging tool dubbed cvslog2web.

cvslog2web is similar to the CVS commit hooking and log info to RSS techniques we’ve seen before, but offers it in one convenient package and adds a few more goodies. In addition to an Atom syndication format, you also get recent history as HTML, permalinks to individual log entry pages, and some careful thought on compacting/summarizing the log entries.

CruiseControl Yahoo Widget

Ganesan Janarthanam got inspired by the idea of providing continuous integration feedback using a lava lamp and created the CruiseControl Dashboard Yahoo Widget. It lets you monitor multiple projects with an animated lava lamp.

When Build Silence Is Golden

Andrew shared the following build frequency chart:

We see a flurry of good builds, then a six-day dry spell where almost no builds take place. Did somebody accidentally unplug the continuous build machine? Were all the developers at a conference?

Perhaps there’s a better explanation when we remember that a build is only triggered when code is changed. In this case, no changes added up to good news for Andrew’s team:

: We are particularly proud of the blank section as it signifies zero-defects on this in-production product ;-)

Congrats!

Another Build Frequency Inkblot

I’m glad to hear that someone else peeked under the CruiseControl metrics tab for a glimpse at their project’s build frequency. Here’s an interesting (colorful!) one from a reader who shall remain anonymous:

Notice how different it is from the first team's chart. For starters, the broken build frequency seems high, but at least they’re periodically getting back to a healthy state. I’d suggest they could do a bit more local testing before checking in code, or perhaps just get a bit more sleep. The person who submitted the chart included the following commentary:

: We’re not so hip on the 40 hour work week; this activity is all from four people in the same office. Please don’t mistake that as bragging.

What else do you see when you look at this chart?

What's Your Build Frequency, Kenneth?

In response to the source code activity charts, Robert Watkins kindly sent in this example chart displayed free of charge in the CruiseControl metrics tab.

It shows the frequency of builds which, of course, are triggered by commits to the source code repository. Looks like a slick way to also monitor trends in scheduled build breakage. What does your build frequency chart look like?

Visualizing Commit Trends

Take a peek at these example charts of source code repository activity. What do they imply? I have no idea, but I find them fascinating.

I wonder what the charts would look like on a project where the team builds things in small chunks (where small means "measured in minutes") and checks in code directly after each thing is built and tested. Or would that kind of predictable development rhythm just make the data less interesting? At any rate, it would be interesting to use a bit o’ automation to keep a running chart from start to end of a project.

Oh, and please be careful not to read too much into these charts. I shudder at the thought of some pointy-haired manager using them as input to an employee bonus formula.

Quick 'n Dirty RSS

Email is still the first thing I check in the morning, but my RSS aggregator isn’t far behind. It monitors everything I’m really interested in. And if a site doesn’t have an RSS feed, I’ll forget to check it.

Trouble is, not all sites have an RSS feed (yet). Beg if you must, then do it yourself.

Tracking Bugs, Cradle to Grave

Rich Freedman wrote in with an overview of the tools his team uses to automate the traceability from a bug report to the file changes that squashed the bug. He writes:

We’ve made the bug-fixing process more transparent and bug-fix code reviews much easier to do by integrating CVS, CVSweb, Bugzilla, and JSPWiki. Here’s how it works:

Programmers put "Bug: xxxx" in the first line of the CVS commit message. A Perl script updates the specified bug report in Bugzilla with the programmer’s name, the commit message, the file names, and hyperlinks to CVSWeb that display the diffs for each file committed. This allows the project lead to access the diffs for any bug fix directly from Bugzilla.
We created a Perl script that runs on the same server as Bugzilla and, on demand, creates an RSS file containing information about updates to Bugzilla for a specified project and number of days (we default to the last 10 days). We then used a JSPWiki RSS plugin that we had previously written to display this information in each project’s wiki page.

This is another great example of effective automation made possible by connecting existing tools with little dabs of glue.

Ruby Growls

And what if that program you want to make growl isn’t running on a Mac, but you’re parked in front of one? Well then, you need Eric Hodel's ruby-growl:

: Growl’s existing Ruby bindings only work on Macs, and I wanted my non-Mac machines to be able to send Growl notifications. I peeked inside Growl’s remote-notification capabilities and wrote up some code that uses UDP to send Growl notifications across the network.

With ruby-growl you get a Ruby library you can use to send Growl notifications from within your Ruby applications and a command-line tool similar to growlnotify.

Grr!

Using Growl for Long-Running Builds

Michael McCracken has another neat Growl trick for those of you with a Mac:

I have a few projects that take a long time to build. (C++ mostly, with many nested makefiles.) Since I’m guaranteed to go check email while it’s building and forget about it, I have gotten in the habit of using this incantation to have Growl tell me I should get back to work:

  gmake; echo "Returned status: " $? | growlnotify "Build done"

This gives me a notification titled "Build done" with the exit status from the shell variable $?, the status of the last performed shell command.

Then he goes on to show you how to plot build times with gnuplot. Nice! Free tools, priceless feedback.

(Growl is good stuff for monitoring things right from the comfort of your desktop. For example, I use it regularly as a tattletale for web sites.)