Writing

Using protocol buffers with no drama

After building more than a few systems using Protocol Buffers in the past couple of years, I've become rather fond of them. They let me do useful things without too much trouble and basically stay out of my way. Sure, it's from You Know Who, but it's some of the "old school" stuff which had the right people behind it. It's also "escaped" by way of an open source release, so it belongs to the world now.

It should come as no surprise that I still use protobuf to get things done. This made for interesting times when I started switching from make to my own build system, since the usual tricks do not apply. With protobuf, you start with your definitions in a .proto file. The "protoc" tool then transforms that into a pair of files ending with.pb.h and .pb.cc. Then you have to compile and link it in the usual way.

My project was built to look for #include "foo/bar.h". Given that, it opens that file and analyzes it. It also looks for a matching foo/bar.cc, and analyzes that as well. This works just fine for ordinary code written by me, but it completely fails for generated stuff like this. The reason is simple: I don't store the output from protoc in my tree.

So, I've extended my build system. Now, an #include "foo/bar.pb.h" will be recognized as a protobuf, and it will run protoc for you. It then directs protoc to write these files into a "genfiles" path, since it is neither source code nor per-build-target output. There is no difference in how you call protoc for debug, default, coverage, or optimized builds, in other words.

In practical terms, it means you can add protobufs to C++ code really quickly and without much fuss. Recall from last week that inputs and outputs had been split, and I created a really simple file called main.cc as a test. Let's take that and extend it to use protobufs as a demonstration.

So here we are in the tree...

~/depot$ find
.
./.depot.root
./src
./src/core
./src/core/main.cc
~/depot$ 

I'll build "main" right up front to give a reference point.

Okay, that's nothing surprising. Now let's create our protobuf definition.

~/depot$ cat > src/core/demo.proto
message Song {
  optional string title = 1;
  optional string artist = 2;
}
~/depot$ 

Right, so now I amend main.cc to #include the header which will be created and then actually use this new "Song" class. Now it looks like this:

#include <stdio.h>
#include "core/demo.pb.h"
 
static void do_stuff(int i) {
  if (i == 0) {
    printf("hello from main\n");
  } else {
    printf("goodbye\n");
  }
}
 
int main() {
  Song rio;
  rio.set_title("Rio");
  rio.set_artist("Duran Duran");
  printf("song debug info: %s\n",
         rio.DebugString().c_str());
 
  do_stuff(0);
  return 0;
}

Easy enough. Let's build it.

Our protobuf has been detected and turned into source code, and then that was compiled into an object. That object was then linked into the binary, so let's run it and see what happens...

~/depot$ bin/core/main
song debug info: title: "Rio"
artist: "Duran Duran"
 
hello from main
~/depot$ 

That's it. If I want to use this same definition in another program, I need only #include it in the same way and start working. Instead of having to worry about tracking all of this stuff in a Makefile or similar construct, my build tool will just find it and figure it out for me.

This sort of thing is useful to me. If I'm cruising along and working on something which calls for a protobuf, I can just write a quick definition, add the #include line, then start using it. I don't have to push all of that state onto my mental stack to go off and deal with build scripts or wrangle Makefile rules.

When it comes down to it, you're either writing code which furthers the cause of solving whatever problem you currently have, or you're off shaving yaks. Which one really deserves your valuable time?

Sorry, yaks. You're going to have to stay furry this summer.