Writing

Feed Software, technology, sysadmin war stories, and more.

Tuesday, October 16, 2012

Just how big is a /29 subnet, anyway?

Here's a potentially tricky question I provided for someone who wanted some fresh material for interviewing: How big is a /29? (Yes, assuming IPv4 here.)

The answers you get can tell you a lot about a candidate. Someone who's a binary nerd will probably scribble down a quick "00000111" (or "11111000", depending on which router planet they're from), squint at it, see that it's just 2^3, will respond with "eight". Someone else who's done this a lot and has the numbers burned into their brain might just lob the "eight" back at you without even thinking twice. They're thinking just in terms of how big it is, and not how many might be usable.

That's a fine answer, but there are others you might get. Someone else might say "five", since they got a static IP assignment from their DSL provider, and that's how many usable addresses they had. They got that number by subtracting the network and broadcast addresses at either end, and then also taking out one for their ISP router's interface in that network. 8 - 2 - 1 gives you 5. Easy enough.

Another person who looked at the bits and who knows about the net and broadcast addresses but hasn't had to deal with a bridged network where an ISP has a "foot" in it might say 6. It's just the above situation without the final "- 1", and that works too. In this scenario, it's interesting to ask how that network gets routed and see if they start talking about point to point links.

Let's say you explicitly ask for "how many are usable" as a followup question, and you get the "6" or "5" answers. As per the above, those can make sense. What if you got "8"? Is the candidate full of it? Maybe, maybe not. Check this out.

A trick we used to do back in the web hosting biz was to use the entire subnet when assigning additional IP addresses to customers. Here's how it worked. First, you might have a /24 allocated to a block of customer space. Those are all "primary" IPs, and so one box is x.x.x.10, another is x.x.x.11, then x.x.x.12, and so on. The hosting company's big router has an interface in that network at x.x.x.254, and the hosts use that as their default gateway.

Now you have a totally different block of addresses that is only used for "additionals". This is when a customer wants more addresses on their box for whatever reason. SSL was the usual case (due to the lack of SNI), but there were other, sleazier reasons like evading IP-based spam sender lists.

The customer with box x.x.x.10 wants 8 additional IPs. We'd carve a /29 out of the additional IP space and would add a route to the internal fabric. This would route y.y.y.192/29 (the new block) to x.x.x.10. Whenever traffic for that network would arrive at the router, it would do the usual ARP request for x.x.x.10 if necessary, and then would lob the traffic at whatever hardware address it got back. It was up to the customer's box to do the rest.

Here's where the magic happened. On a Linux box, you could configure a bunch of aliases on your network interfaces and give them 255.255.255.255 netmasks (aka /32). With this in place, the machine would happily answer to traffic arriving for any of those addresses. You'd just add all eight of the additional IPs (y.y.y.192 through y.y.y.199) as separate aliases and that was it.

The Linux box didn't know they were "really" part of a /29. At this level, it didn't matter. They were just interfaces it happened to have, and there was traffic which just happened to arrive for them. It would service that traffic like any other.

I was told by the Windows techs that the same trick didn't work for them, so apparently they had to burn two addresses per subnet and there wasn't much they could do about it. This had some interesting side-effects when special situations came up.

One time, I saw a case where there were a bunch of additional IPs on a Windows box and none of them were working. This was some kind of "special" customer, and this person was screaming bloody murder because their machine wasn't working. They were calling the president of the company, who called the segment VP, who wound up getting the shift lead involved, and eventually it rolled downhill to me. I wasn't a Windows tech, but I "knew networking", so he came and grabbed me to have a look.

Just from looking at the addresses in the provisioning system, everything seemed okay. I could kind of eyeball them at that point and could see the natural alignment of various subnets, and it was all good. Nothing about it looked out of place, and if I had put it on a Linux box, I'm sure it would have been fine. I spent some time writing out the IPs by hand to make sure I wasn't seeing things, and again, everything seemed great. The ranges made sense, the netmasks made sense, and everything looked like it should work... but it still didn't work on that machine.

At the time, I didn't know about the limitations they had for additional IPs on their Windows installs, or what the techs had been doing to add them. If I had, it would have explained a lot.

It turned out they had this tool which would take the lowest and highest addresses from a block and would add everything in between. This saved them a whole lot of monkey work clicking through IP configuration screens. Whoever got the ticket had given it the first IP address of the bottom /29, and the last IP address of the top /29, for a total of 16 IP addresses. Whoops.

The script proceeded to add *ALL* of them, including the part in the middle where the bottom /29 had its broadcast address, and then the top /29 had its network address. Neither of those were valid, and so Windows threw up its hands and refused to accept the configuration.

Somehow, one of the guys in networking had seen this before, and knew that it tended to happen when a customer got two adjacent blocks allocated for additionals. He took one look at the IP list, dropped onto the machine, removed the two middle addresses, and everything was fine.

I suspect that if they had added them by hand in the GUI, it would have flagged it as invalid and disallowed it. Instead, since they had some tool which basically hotwired the addresses into the system (without the same level of checks), they were able to create a broken situation.

Thinking about it now, I wonder if the two /29s had been aligned such that they could have been rolled up as a single /28. That would have allowed for 14 usable total instead of the 6 and 6 they finally got.

So, how big is a /29, exactly? It depends!