Software, technology, sysadmin war stories, and more. Feed
Monday, April 2, 2018

Newer C++ features can create a lot of system yak shaving

I've been slowly easing myself back into the world of programming on my own time now that new things I would create are no longer encumbered by a certain former employer. Doing this has exposed some other problems that have created a whole lot of yak shaving for me.

First, there's the matter that C++ is no longer holding still. For a while, it was the 1998 flavor and that was that. I didn't actually start using it in any real quantity until several years after that point, and by then, most systems I would encounter had decent support for all of it.

This is the situation I had seven years ago when first striking out on my own to pursue random interesting things. The result was a code base which was written with nothing too fancy, and it would compile just about everywhere: various flavors of Linux (x86, x86_64, ARM), my Macs, the BSDs, and so on.

That said, there were a few "add-on" conventions I had picked up from working in the field. I had started missing them in my own projects, and brought them into my projects by re-implementing them. One of these was a specific flavor of smart pointer called scoped_ptr which worked just like the one at a certain company. It gives you a few neat things, like the ability to just forward-declare a class inside your .h file and not include the whole thing right there. This makes your dependencies simpler and compiling tend to be faster, too.

It also means you don't do a whole lot of bare calls to "new" and you never really call "delete". The smart pointer hangs onto the actual pointer, and its destructor calls delete for you. You can't leak it if things exit correctly.

Since then, C++ itself has picked up both std::unique_ptr which behaves as the old scoped_ptr did, and std::shared_ptr which does some interesting reference counting stuff. (I'm deliberately skipping over std::auto_ptr, which is gone now anyway, and all of Boost. Ugh.)

During my most recent stint in the "real job" realm, I started using these newer features. Besides the native smart pointers, there are also neat things like simpler ways to flip through a container. Whereas previously you might have had to create a foo::const_iterator, and then do something like this...

for (i = f.begin(); i != f.end(); ++i) { ... }

... now you can just do something like this ...

for (const auto& i : f) { ... }

Pretty great, right?

I was wary of shiny new things in programming languages in part due to a post on the very topic from 2012. In it, I thought that compiling things to binaries would keep you safe. I hadn't encountered the likes of C++ versioning yet.

Still, while working inside the company, all of that business was handled for me, and for the most part, it Just Worked. I ignored my own feelings and jumped in with both feet, using lots of the new features in my projects, and enjoyed the overall simplicity.

Now, back on the outside, I find myself looking at my existing code base which has been nearly static for almost five years. There are things I would like to change, simplify, or in some cases just fix. My natural inclination is to start using the newer features while doing this work, but actually going there could be troublesome.

Since all of this lives in a single source tree, anything common has to compile and run on all of my machines. In particular, if anything gets used in a low level library (like logging), then it has to be maximally compatible. Either the source lays low to avoid causing problems, or every single machine has to be dragged up to the new baseline.

At least one of the machines is based on RHEL 6, which comes with a version of g++ that doesn't even do C++11 stuff. It will throw all sorts of entertaining parse errors if you try doing any of these 'auto' tricks on it.

I have to admit this was a real bummer. I know better than to try to upgrade anything that important on a Red Hat type box. There's just no sense in attempting to diverge from the base system. You either keep it or throw it out and go to the next major release... and I'm not ready for that. That leaves me with one option: leave it in place and then "side load" a new compiler and/or libraries, and then switch to using it for all of my local stuff.

A couple of days ago, that's exactly what I did. It turns out that it's not a huge deal to compile gcc with a prefix which will put it off to the side and then just drop the entire mess into that directory. You can then just do /path/to/your/new/g++ and you'll have the new compiler working for you. It'll gobble up the new syntax and will be happy.

Before you declare victory, there's a major complication which will arise: it's still going to (try to) use the stock libraries on your machine, including libstdc++. Yep, that's right, sometimes not all of C++ gets compiled into your program. At least some of it stays on the disk and gets dynamically called up when you start it. You might not notice this at first, but once you start trying to use some of the newer features (like std::unique_ptr), you will eventually smash into a wall where it complains about some symbol like CXXABI_1.3.9 not being there. Compiling and linking will look fine, but when you actually try to run the new binary, then it'll blow up.

Now you get to play the "oh, I want you to use the new libstdc++ over in some weird separate path" game. This is when you start playing with "-rpath" (which itself starts getting mighty spooky when you start playing with $ORIGIN) or maybe you try "-static-libstdc++" to drop the dependency entirely (and grow the binary accordingly).

Maybe instead of that, you start thinking about LD_LIBRARY_PATH. Perhaps the idea of building your own chroots starts appealing to you. Just think of it: every program built from that code base now winds up running on a wacky little mini-me environment. Of course, if these programs are supposed to run from the command line like anything else on the box and no fancy wrappers, those aren't great solutions, either.

So far the best thing I have is just passing rpath to the linker and pointing it off at the same directory where everything else was sideloaded.

Of course, once you do that, then you find out that existing C++ libraries on the box are no longer happy being linked into your program, like, oh, say, protobuf, gmock or gtest. Why? For one thing, std::string is almost certainly different and it won't match up. There may be other issues, too, but the point is clear: this is not going to happen.

The only way out of this is to tear off the bandage and rebuild all of those with the new compiler and side-load them as well. When it comes to C++ programs and libraries, you really do not want to try to mix and match across compilers. It really does not like it, and it will make you suffer.

What a colossal mess.