Writing

Feed Software, technology, sysadmin war stories, and more.

Thursday, September 13, 2012

My love-hate relationship with development on Macs

I've been trying something new as an unusual way to keep tabs on how well some software is running. As mentioned in other posts, I run a web site which receives a big chunk of radio spectrum, slices it up, decodes the audio, and saves it out to MP3. This thing runs continuously, and it chews a fair amount of CPU power. This also directly translates into electricity. As a result, I have a "Watts Up?" meter in line with my computer to get a feel for how much work is really happening. Why bother? It's complicated.

Recently, there has been an explosion of projects using the $20 "RTLSDR" TV sticks which can also pull in significant quantities of raw RF energy. This gave me a way to mess around with fine-tuning this stuff without affecting the production system (a rather expensive USRP). So, one of the things I've been working on is reducing the amount of CPU power which is required to keep up with the stream.

As it turns out, the most convenient place to do this is on my Mac Mini which is in a good place to snag radio signals coming in my window. It's small, quiet, and doesn't pull much power when idle. I can't say that for my "cold spare" Linux box, so that's out for extended runtime/burnin tests. My woefully underpowered Emachines laptop is too slow to handle this kind of work, so that's out too.

There's more to this, though. To actually use one of these TV sticks with GNU Radio, you need a relatively new version of it. The version of GR in Macports is too old to work, so it means treading into the realm of (gasp) building it by hand. I also had to be careful to not have it spew stuff all over the place during 'make install', since there's no local equivalent of "makepkg" (Slackware) or "rpmbuild" (RH and friends) here.

It took a couple of hours for all of this stuff to download, compile, and install, but I finally got a working install which stays confined to its own little pseudo root. DYLD_LIBRARY_PATH (the Mac equivalent of Linux's LD_LIBRARY_PATH) makes it all Just Work in spite of the unusual install directory situation.

Finally, I could get down to the business of working on my scanner code without fear of disrupting the production site. It's been interesting to see how certain bits of library code can lead to horrible tight loops which add dozens of watts on the meter. I've been trying to rework things with an eye on the meter while not compromising on reliability. It's been an interesting challenge.

I would hope that someone would say "why not use strace to look for excessive syscalls", given my love of that tool. For those who haven't had to suffer through this kind of work on a Mac, let me just say right now: there is no strace. The closest thing you get is dtruss, and while it shows you certain things, it is a pale shadow of its penguin-riding buddy. Forget about strace's cousin ltrace. It's just not there.

Someone else might say "what about gprof?", and that's also a good suggestion, but again, this is a Mac, not a Linux box. Asking about it or searching around will eventually lead you to Xcode and Instruments. Again, it's a far cry from the things which have existed for years and got the job done.

Still a third interjection comes from the crowd: try the Valgrind suite! This is normally an excellent idea, and I love any chance to use their wonderful tools, but again, the Mac makes this harder than it has to be. Valgrind only recently updated for Mountain Lion support, but per their 3.8.0 release notes, it is "not usable for serious work at present".

Macs are a dangerous place to work on this kind of low-level plumbing stuff. You can get surprisingly far by pretending it's just like a Linux box, but eventually you may find yourself in a situation where you need Really Deep Magic to sort something out. I typically wind up punting the problem over to a "real" machine where I can get inside of it and find out what's going on, assuming the problem is my code and not the kernel or C library.

Because, you know, the stock Mac C library never has problems. Oh no. Never.

So yes, sometimes I optimize my code using a power meter.

Power meter

Or, in meme terms, optimizing code using a power meter isn't normal, but on Macs, it is.


September 14, 2012: This post has an update.