It's been two years since I escaped from that place in Mountain View to start my own thing. I'm obviously still writing, and still immersed in the world of technology, with various nerdy creations issuing forth from time to time. At the moment, I'm working on the second book so it's stealing cycles from things like software defined radio projects.

I think I might purposely take a trip down to that office space where I encountered a distinct lack of connectivity. It's amazing how much you can get done when you don't have viable routing to the Internet. By way of comparison, last year, the book was a smaller project, and I managed to finish a bunch of it while flying to Maine and back. When you have to fly from San Jose to Denver to Charlotte to Portland just to get there, and from Portland to Charlotte to Phoenix to San Jose to get back, there are many opportunities to buckle down and just work without distractions. Sorry, HN and reddit, but it's true.

...

I've gotten a few anonymous feedback questions and requests over the past couple of days which need responses, so here we go. (I can't e-mail these people since they didn't leave any sort of contact info.)

Last week, someone asked if I would "consider doing a short tutorial on AJAX with C", that was "... fancy like a dynamic datagrid". Okay, that's an interesting one.

Assuming you really mean C and not C++, I suppose I could. I could certainly write something which is just plain old C and doesn't use any part of C++ to get things done. The biggest impact I can imagine at this point would be from throwing out the STL and strings, and not being able to organize things in classes.

None of this would stop me, particularly since I've done this sort of thing before, but it would definitely slow me down. In order to do this in C, I'd probably need to do things like writing my own hash table implementation in order to store the parsed results of the query string (the part after the ? in a URL in a GET situation) or the POST data received on stdin. If not that, then I'd have to make it parse into a temporary space, and then call back into the C code, which would then have to figure out what to do with it.

Basically this would mean my calling code would pass in a function pointer and that module with its callback function would be something like this:

#define CGI_VAR_SIZE 256  /* or from a .h file... or something... */
 
static char cgi_user[CGI_VAR_SIZE];
 
int cgi_callback(const char* var, const char* val) {
  if (strcmp(var, "user") == 0) {
    strlcpy(cgi_user, val, sizeof(cgi_user));
    /* ... and maybe check the retval from strlcpy and bail... */
 
    return 1;
  }
 
  /* ... over and over ... */
}

... Only, I'd have to fight with whether 'strlcpy' even existed on that platform, and then either pack in a suitably-licensed copy or (ugh) write my own. Then I'd need to wrestle autoconf into noticing such things and have it do the appropriate #define and/or Makefile creation magic to make sure the compatibility objects were compiled and linked into my final creation.

This also assumes that everything I'm shoveling around in there can be expressed as '\0'-terminated C strings. If any binary stuff is involved, now I have to start tracking lengths explicitly and pass all of then around too. Maybe I'd have to invent my own "string". UGH.

In short, it would be a serious piece of work. I already decided that Make and autoconf are things to be avoided, and so avoid them I shall. Sorry.

Now, if you're okay with C++, most of that work is already done. The AJAX part would be a matter of some jQuery (or raw JS, perhaps, but why?). The real unknown would be the "dynamic datagrid" part. I had never heard of it before, and had to dig around to find out this is some kind of ".NET" thing. It looks like a fancy way to render a table of data, sort of like a program like Excel might use.

So really, if you want to see a web page with some tabular data which is dynamically populated via AJAX, that wouldn't be so bad. The last big questions are: what sort of data, and is it read-only, or read-write?

If you want to see this or have other requests or comments, let me know. Thanks!

...

Another anonymous reader asked about my encrypted time capsule post from last summer. They want to know where I got the screenshot.

The answer: I took it myself. I just cut out the name of the device since that's a private detail and doesn't affect the point of the post which is that "encrypted" now appears.

If it looked weird, that's just because I cropped the screenshot to show just the pertinent part. The dialog box itself is too wide to fit in a post here without being squashed. I put a copy right here in this post to show what I mean by squashed.

...

In my "No." post about the GNOME terminal bug which was closed with just one word and "WONTFIX", I mentioned not knowing anything about whether there were "out of band" discussions about this problem which didn't show up in that bug (698544). In response, someone wrote in to give me some details on the situation, including a link to another bug with some more context (695371):

None of the 'usual' mailing list or blog discussions along the lines of 'does anyone actually need this feature any more, it is making it hard to do X, Y or Z' have happened. See below.

https://bugzilla.gnome.org/show_bug.cgi?id=695371#c9

So, okay, apparently the person who closed the second bug has already expressed doubts that it's of any use. Given that, it's not too surprising that the feature was yanked and there seems to be no interest in restoring it now.

In pondering this situation, I've attempted to think of some experience from my own past in which I deleted a feature from something and received pushback from users. The problem is that I can't think of any. I've written software which has had real users over the years, and while most of it is "plumbing" stuff which doesn't have a shiny user interface, all of it still had features. Some of those projects went through different stages of life and a couple of them wound up being ported to some other system or effectively rewritten as part of a bigger coordinated change.

I'm having trouble remembering examples of this. About the only thing which comes to mind is a project I did with some other people some number of years ago. I had to purposely exclude some modules from later releases since that side of the project had fallen behind. I figured it was better to only ship code which was still being actively maintained rather than trying to maximize some feature list.

The difference is that nobody was forced to upgrade. They could stay on an older version (with the now-unmaintained code) as long as they wanted. Unlike things like GNOME, there were no complicated dependencies. You could easily hang back a major release or two if needed and nothing bad would come from it.

Maybe that's the true problem here. These interwoven dependencies mean that you are forced into upgrading even if it means giving up something you really want to have. The only solution to this is better isolation and stronger interfaces which stand their ground and don't change every time the CADT model strikes. Then users can treat such stupidity as damage and just refuse to upgrade.

For what it's worth, that is, and continues to be, my response to iOS 6.

...

Finally, I had a thought about "No." from the GNOME situation. It's a movie from 1973. Perhaps you have seen it, too.

Battle for the Planet of the Apes.

He uttered a negative imperative.

Sometimes, they'd pick someone random on the floor to "be the boss" and they'd transfer the call over there. That person would do their best calm and measured voice and would take the abuse from the caller and would calm them down. The caller usually just wanted to get the original tech in trouble. This generally happened if no actual managers were available. On second and third shifts, and particularly when manager type people were on "offsites" to places like Las Vegas, sometimes we'd be on our own for a few hours at a time.

The other one I heard about was one that actually has a far simpler resolution. You can basically shut them down cold when the customer reaches for this particular trump card. Here's how it works.

First, the customer gets angry for whatever reason. Maybe they have a good reason for it. Maybe they don't. A handful of people who never intended to pay us (fraud customers) used to do things to get credits out of us to negate their bills.

Second, the customer says something like "I'm going to sue you" or otherwise threatens to summon his land sharks.

Third, this part is easy. Like I told my friends, you aren't the company legal department and you aren't authorized to speak on their behalf. In fact, you now have a responsibility to shut down this call and refer them to legal because they have now escalated the matter out of any realm where you can meddle.

The trick, of course, is that in the middle of the night (second and third shifts, remember), there are no lawyers available. There might be someone on call for really crazy things like when the feds show up with a warrant (and yes, this happens), but they sure aren't at the office. All you can do is take the customer's contact data and forward it on, and then shut down the call.

Usually what happens at this point is that the customer realizes they have just hosed themselves. Now they have to deal with an actual lawyer, and tech support will no longer do anything for them. If they had an actual technical problem, their chances of having it resolved in a reasonable amount of time just went out the window.

While I never had this happen to me and thus never had a chance to use it, I can only imagine what would happen next. I suppose a customer who didn't really mean it would eventually cool off and try to apologize to whoever they had on the phone. They might convince them to start treating it like a technical matter again instead of being locked up in "nope, it's a legal matter now" mode. They might have to call back and play tech roulette and hope to get someone who doesn't know about what happened on their earlier call.

If there's a moral of this story, it's something like this: if you threaten to whip out your lawyers on some poor technician who probably has never even seen the inside of a courtroom, don't be surprised if you find yourself unable to get actual technical help for a long time.

First, the arcane bit of knowledge. If you've ever worked with SSL certificates, you might have noticed a few of the traits which are set by the signer. In particular, there's an expiration date which sets the last time it's valid, but there's also a "not before" time. This typically is used to set the period of validity to an exact span of time instead of having no defined beginning.

This might bite you the first time your clock gets messed up and you travel into the past. When this happens, all sorts of secure-side web sites will stop working because you're running in a time before their certs will be valid. Not all of them have the same span of validity, so it might work for some and fail on others depending on how far back your clock has slipped.

So here's where that knowledge runs up into reality. In my role as de facto tech support monkey for the family, I get to hear about machines which aren't playing nice. Last time, it was about a stupid wireless switch on a laptop. This time, the report was simple enough: "the machine says it's set to before 2008, and I'm getting this guest network thing".

This was a Mac, and for whatever reason, it had decided to reset itself back to 2000. When it came back up, this apparently caused it to render some or all of the keychain inaccessible or otherwise invalid, including the wireless network password. I figured this was something stupid like "not-valid-before" behavior, and it would need a valid date to start working again.

With no network access, it was unable to do the usual NTP magic to sync its clock, and thus it was not going to heal itself. The "solution" was to disable NTP sync, manually set the time ahead to the approximate current time, and then switch to the right network. That actually worked, and I was able to jump in remotely to take care of a basic sanity check on the machine after that point, including re-enabling the usual NTP behavior.

As for why it popped up the guest network "walled garden" thing, that one is easy. Someone had previously associated this machine with a guest network at that location, but it was a lower priority than the usual secured network. When it failed to get on the secured network, it tried the next entry, and the walled garden detection on the Mac popped up the login screen.

It was just one big bag of weird.

What's really annoying is that I've actually been bitten by this before, and I didn't have any idea what was going on. I wound up having to reset a bunch of passwords to get things going again.

It's the combination of that past event and fiddling with SSL that suggested the right course of action today. Does this sound like the sort of thing a normal Mac user should know?

It sounds suspiciously like the sort of Unix wonkiness which has kept me employed for many years. These things tend to leak through their every attempt to make the system look nice and shiny on top. I know it's really ugly and greasy underneath. Brokenness like this only reinforces it.

One approach is just to store every username and password you've ever seen. It's simple, it's easy, and it's a great way to really let your users down. Storing things in plaintext just makes life really interesting when your systems are compromised and the list of accounts and their passwords is spread around.

If your database looks like "user | user@example.com | password" then it might just be built around this sort of design.

It seems the next step up from that is to take a page from the world of shadow passwords and use some kind of hashing scheme on the password. Then, instead of storing the plaintext, you just store the result of some kind of black box function. In order to verify a login attempt, you just take their password, run it through that same function, and see if you get a match.

Of course, those systems get compromised as well. The lists are obtained and then the hashing function is determined, and then a series of brute force runs of that function eventually turn up the probable plaintext. It's possible to make educated guesses about what a password might be based on the username or other components of their profile. I think tools like "john" will grab bits of the "GECOS" data from the passwd file to help out. Users who had things like "District Office" in their location data would have a password of "district" cracked right away, in other words.

So about two weeks ago, I ran across a paper which talks about creating "honeywords" in the hopes of detecting passwords which have been cracked. It seems to be based around the notion of having passwords which might be valid hashes but aren't actually useful for accessing a given account. If some evil attacker sends in the wrong combination, it might be able to guess that someone's been messing with a copy of the hashed data.

It was pretty late that night when I found this, and that's probably what lead to my half-baked idea which would add to this. You keep the notion of having a bunch of bogus matches, but you go far beyond it in terms of enlarging the problem space.

First, instead of just hashing passwords, hash the usernames too. Then, instead of having separate lists of usernames and passwords, have just one. Third, instead of only having hashes for actual usernames and passwords which are in use, have all of them which have ever been used, including the ones which aren't even valid any more. Finally, hash a whole bunch of random crap and put it in there. Mix it up really well so there's no pattern to which is "filler" and which has actually been used by an actual user.

Let's say your system has 1000 active users, and each one has had two passwords on average. That would normally represent 3000 hashes. To make life interesting, we'll push it up to about 30 million hashes by hashing all kinds of stuff: random strings, dictionary words, whatever. Then we mix it up so it's not apparent which rows are which.

The table is simple enough: a row id which is unique, and a hash which is also unique.

Obviously, we need some way to identify which of these combinations is valid. That part is simple enough: it's just a tuple. You might say that ( 123, 456 ) is a valid user. That is, hash #123 and hash #456 together are a valid user. It might be necessary to get some kind of user ID back from this, so in that case, ( 123, 456 ) = user 1337. Whatever.

If you store this mapping in the same place as your hashes, you're still in the same boat as before, so let's make things even more interesting by storing these tuples somewhere else. Put them under separate cover as a completely different hosted service elsewhere in your organization. That service would only store these tuples -- just the simple mapping of ( 123, 456 ) to 1337, and so on.

Without the hash table, the list of tuples is just a bunch of numbers. Without the list of tuples, the hashes is a giant haystack which is full of plausible needles, and they're all encrypted. You have to brute-force all of them to get back their plaintexts. Only then can you start guessing combinations, and this time, you don't know the order, since how can you tell if it's a username or a password?

Okay, maybe "rachelbythebay" looks like a username and "hunter2" looks like a password, but who's to say that it couldn't also work the other way around?

It would be necessary for the tuple service to be vigilant about traffic patterns, and try to flag those which seem dubious. In that sense it might be possible to notice something bad before it goes too far.

I wonder if it would be useful for some third party to provide the tuple lookup service. It would be one fewer thing to reside on the systems where the login attempts are taking place. Of course, given that, the whole job of storing hashes could also be outsourced to yet some other provider as well.

If both of them were stored offsite, then a login attempt would be pretty interesting. First you collect the username and password, hash them, and ask the hash server for the numeric values for those hashes -- their unique row IDs. Then you take those numbers and fire off a request to the tuple server to see if they are valid. If so, then you allow the user in and proceed as before.

I suppose you could also send a bunch of "fuzz" requests to both the hash server and the tuple server so even they wouldn't know exactly what was going on. That would probably defeat the notion of having those providers run traffic analysis on your queries to find attacks, though. It would be back on you to do that locally.

I'm no crypto person, so this is probably full of holes and has probably been hashed (heh) to death somewhere already. Just in case it's something fresh, I'm putting it out there to hopefully stir some imaginations. Maybe something useful will come from all of this.

Imagine a laptop computer which has a touchpad for mouse type input. This is probably any laptop you can buy these days. You use it instead of having to cart a separate mouse or trackpad around. That's not too special. What's special about this laptop is what else the trackpad can do.

I'm not talking about multi-finger gestures. Assume it does any kind of gesture your heart desires. This is about user recognition. You see, this trackpad can somehow figure out who's touching it at any given moment and will switch to their workspace to let them do things.

If I'm using the machine and then pass it to my friend who needs to look something up, she can just tap the touchpad and it will immediately (this is important!) jump into her session. Her web browser and other stuff will be there just like it was the last time she used it, and it will respond as if she had been using it all this time.

If she hands it back to me and I tap the touchpad, it will immediately jump back to me. This can go back and forth indefinitely.

Now, given this constraint, how would you design this? Would you just run my tasks and her tasks in parallel in a Unixy sort of environment and just switch them in and out as the user presence data changes?

Just how quickly could you do this on (say) OS X? How about Linux? How about Windows? I'm guessing tens of seconds would be required to make it finish switching to another user. It would have to page in a bunch of read-only binary data which had been evicted from memory. It might have to bring user data back in from swap which had also been evicted at some point.

This is a slow process. It would be evident that you did not really have two smooth and responsive environments at your disposal. You really just had one environment, and with a big lag, you might be able to bring a second one in to replace it. Then you have to expect that same amount of lag when switching back.

The system would have to be explicitly designed around "fast flips" for it to be instantly responsive. You'd have to go to lengths to make sure that anything which could be a user-interactive process remains available at all times. This might mean locking pages into memory or even coming up with some completely new marking, allocation, and eviction plan to handle what can and what can't be swapped or paged out.

If you tried to do this with existing systems, you would experience only pain. It would be obvious that it was something layered on top and it would only work if you threw colossal quantities of resources at it. You'd need the fastest CPUs, tons of memory, and the absolute fastest I/O devices you could get. It would also be stupidly expensive as a result.

And still, since it hadn't been designed to be responsive, it would eventually "rust" and would reach a point where it wasn't keeping up any more. Your users would get annoyed by this. Some might "level up" and buy the latest new ShinyThing with even faster hardware. Others might just give up and write it off as impossible.

So now that I've established how "bolted on" technology can be apparent, where do you see it in everyday life? I see it on my Mac any time it decides to run a backup. This makes it evict all kinds of stuff from memory. I assume there's some horrible VM implementation involved which makes it punt all kinds of stuff out of physical memory just because something is reading all over the disk.

I have had multiple occasions when all I was doing was running iTunes to play music and Firefox to play my scanner audio, with it being mixed and pushed to my headphones by Airplay. The machine was otherwise idle, and indeed, it was sitting on my desk not doing anything else.

Time Machine fired up and started doing something. How did I know this? Easy. iTunes started skipping. Then, when I went to poke at the machine, it wouldn't respond for easily 15-20 seconds at a time. This is where you poke the pause button and it does nothing, so you wonder if it didn't "make the connection", so you push it again. Then when it finally wakes up, it pauses and immediately unpauses since it handles both of the button presses at nearly the same time.

The same applies to volume changes and everything else of the sort. When this happens on the system, forget about getting anything done.

The system log also gets polluted with notices about how it's killing this process or that due to memory overhead. This is a box where I had to double the memory when I moved to Lion because it had become so stupidly slow doing the same tasks as before. Adding memory didn't fix anything - it just made it suck less.

It reminds me of the days when the Linux VM implementation wasn't quite there yet. They hadn't really figured out a good balance for things like "swappiness" and OOM killing. Fortunately, that was a long time ago, and my Linux boxes haven't done that in probably over a decade.

My Mac, however, likes to party like it's 1999.

So imagine the latest report: someone was unable to get online. She hadn't been able to get online in quite some time. This is a machine on the other side of the country, so during one of my prior visits, I put some remote-access magic on it. It checks in every time it gets online and that time is logged.

I looked, and sure enough, it hadn't checked in for several days. Something was obviously wrong, and it wasn't just a perception issue. That box was definitely offline. I got on the phone and started asking questions. It was powered up and running Windows but it couldn't get out to the rest of the world. Chrome and friends were all complaining about the lack of connectivity.

I happened to remember about where the combination cable modem and gateway lived in that house, so I directed this person over there. As I listened to a description of a device which is unlike anything I've ever owned myself, it all sounded fine. Still, I requested one simple thing just in case it was localized to that box and not the laptop itself. Yep, I asked for a hard reboot of the box by yanking power, waiting, and restoring it. Nothing happened.

This was actually mildly useful since nothing changed on the laptop itself. You'd think that kicking the wireless access point offline would make Windows change states somehow, but nothing of the sort happened. The next step was to get down into the network control panel and start poking around at things. I don't have that particular flavor of Windows here, but some quick searching turned up a few screenshots. My thanks to people who put such things online. It's the only way to direct people when you can't see the screen yourself.

While poking around in here, it became evident that Windows wasn't seeing any wireless networks. There should have been at least one (the router we just rebooted), plus possibly a neighbor or two. Seeing nothing at all gave me an idea. Maybe that stupid hardware switch had been turned off. The question was: where did it live?

I asked for a description of the device: what sort of physical buttons, latches, and sliders are there besides the keyboard itself? This actually wasn't enough to localize the problem, but I did get lucky with a search using the model number and "wifi switch". This is what I found:

Take a good look at this. There's a round button in the center which is pressed regularly to wake the machine up, put it to sleep, or turn it on and off. It just happens to be right next to another round button. This second button is smaller, but they are sufficiently close to where one might be confused for the other.

If you wear glasses or contacts, do you ever sometimes do things without wearing them? I suspect many people do, especially early in the morning or late at night. Now imagine reaching for that button, pushing it, and turning off your wireless adapter without realizing it. The next time you try to use the machine, nothing works, but as far as you know, nothing changed.

I took a guess and assumed that's what happened, so I asked about its color, and it sounded wrong. I asked what happens if it is pushed, and heard back that it had changed color. Then a whole bunch of stuff started changing on the screen (in the still-open network control panel), and unsurprisingly, it got back online.

That's all it was. A strange little button which basically exists to make life difficult for people. I only knew about the possibility of this switch existing because my own cheap Windows laptop has it as a slider on its front panel, right next to headphone and mic jacks. If not for that, I'm sure it never would have occurred to me.

My first laptop didn't have wireless. I had to add it with a little card. My current laptop is a Mac and doesn't have a switch to turn off the wireless. If I want to shut it down, I twiddle a setting on the top bar or dig around in the preferences.

Let's say there's a good reason to make this a hardware switch. Why put it right next to the power button? This is something which is only going to cause pain unless you really want to push it.

Of course, it wouldn't hurt if Windows could actually read the state of this switch and say something about it on the screen. Sure, the user would still have to see it and read it (good luck!) but it's better than nothing.

Not too long ago, I found myself in a quasi-coworking place where you get office space, a markerboard and all of that stuff. It's the sort of thing where you can bring a client and hash out a big project. They provide the bathrooms and drinking fountain and all of this.

One thing they supposedly had was wireless Internet. I assumed it would work, but hadn't actually tested it. While waiting on my client to arrive, I tried using it. I found that it would just sit there and spin forever before finally yielding something ridiculous like "5 0 0 S e r v e r e r r o r". Yes, it actually stretched the letters out like that.

I knew it had to be one of these dumb "walled garden" things, and really just wanted it to get out of my way. All I needed was enough connectivity to fling my VPN packets to my tunnel host. Usually that means starting up a sacrificial browser session and clicking through their annoying page to get unfettered access.

This time, it wasn't happening. All I got was that stupid error page. One thing I noticed during all of this was that I was actually able to resolve hostnames. Not only that, but I could bounce queries off specific servers. Apparently UDP port 53 would get through. I confirmed this by logging into my tunneling host and using tcpdump to watch for traffic. Sure enough, I could emit some stuff on my laptop and see it back on my host.

As for how I got on my tunneling host, well, that was also a stretch. I barely had a cell signal in this weird office place, and was just able to ssh from my phone and kick off tcpdump. It's the sort of thing I only use when absolutely necessary since it's such a pain: the tiny keys, limited terminal space, and horrible packet loss leading to laggy sessions.

It was just enough connectivity to let me add an iptables rule. This is where it gets really dirty. I basically did this:

iptables -t nat -I PREROUTING -s external.ip.of.office.network -p udp -j REDIRECT --to-ports port_of_tunnel_server_daemon

Basically, I took *ALL* UDP from that office and shoved it into the program which runs the VPN/tunnel. That way, if I'm able to find so much as a single port which will pass traffic unmolested, I can get on. It just so happens that port 53 works... some of the time, at least.

It was just enough of a pipe to let me do things like git sync operations. Trying to look at web pages was asking too much of this laggy, lossy connection. It was shades of my 4800 bps cellular modem hack, only far worse.

When I say it's possible to be in the middle of Silicon Valley and have miserable Internet access, I mean it. If I end up working at this site again, I'm going to have to buy my own little cellular hotspot and find a place with decent cell reception. There's just no way around it.

Tunneling over port 53. What a horrible hack.

Looking for a Sprint store? Don't listen to Apple Maps.

There's no Sprint store anywhere near there, but there are a bunch of towers. What do you want to bet it's a cell site?

...

Looking for the stadium? Prepare to be sent to the chemistry building.

The actual stadium is several blocks away and is across El Camino.

...

They have data but they ignore it. See the big grey blob on the green field which is in a square?

That grey blob is the actual Mission Library. The building they've labeled is actually a bunch of apartments. They might have books, but I doubt they're going to lend them to you.

...

New Maple Street doesn't exist any more.

According to historicaerials.com, it's been gone since at least 1998. The fact that SCU built Loyola Hall right there probably has something to do with it.

...

Want to mail something? Don't follow this map marker.

That's a nursing home. It might be nice to go there and deliver things to the residents to make them realize that they aren't alone and people care about them, but the USPS it is not.

...

This one borders on outright negligence and will probably get someone hurt some day:

Kaiser Permanente had a large facility here once upon a time, as evidenced by the road name: Kaiser Drive. However, it closed. They built a new center a mile or two away at Homestead and Lawrence Expressway. All of the offices and services were relocated, and then this site was shut down.

Over the past few years, that site was bulldozed down to nothing. Now, there are streets going in for a new residential community. If you have someone who's desperately in need of medical assistance, do not trust Apple Maps! They will take you to someone's shiny new condo and NOT a hospital.

So, when did Kaiser close this hospital? Are you ready for this?

August 2007.

Shameful.

...

Do you like local history? Do you like pretending the past is still with us? Then use Apple Maps.

Apparently there was a Jefferson School here in the 1960s, but it's gone now. If you go to that site now you'll find Extreme Networks, and if you go behind it to Agate, a bunch of gangbangers. Not recommended.

...

I love when parks are named for people. It's so respectful.

Unless you're Apple Maps, then you misspell their names. Henery? Is that another word for a chicken coop?

...

As I scrolled back over toward SCU, I got a different stadium icon right on top of the same chemistry building they labeled as Schmidt stadium earlier.

It's not a baseball stadium and it's not a soccer stadium, no matter how many times you mislabel it.

...

Want to make friends with the Santa Clara police?

Just follow Railroad northwest past Benton. It'll involve breaking through gates, since that's not a road -- it's the PD back lot! You'll make a nice addition to their temporary holding facility. Think warm thoughts during your cavity search.

...

Do you like tacos and burritos? Don't trust this map to find them.

Taco Bell is across the street. The building they labeled is a hotel.

...

Are you the sort who makes your own roads? They like people like you in Cupertino.

You might think that's an extension of Pierce, but it's just a parking lot for Denny's. Don't cut through there.

...

Once you're done meeting the police, go say hi to the fire department.

Don't drive at that spot. There's a fire station in the way.

...

Finally, there's this bit of insanity which took a while to figure out. This should give you some idea as to the "quality" of the data they're using to feed this thing.

Benne School? What the heck is Benne School? That's an apartment complex.

I didn't stop there. I dug around and found that there was in fact a Bennett School off Lawrence Station Road back in the day. It's gone now. However, if you search around for "Benne School" on the web, you will find countless web pages which insist that such a thing exists, and even has a set of coordinates.

Clearly, Apple has inhaled all of this crap data and is serving up fresh steaming piles to anyone who has the misfortune of using their stock maps.

...

Incidentally, I took all of these screen shots this afternoon, and I used the opportunity to look at the problems I had captured previously. It seems that nothing has been fixed.

Things are broken. Very broken. Things are not improving. If you rely on this for any sort of navigation or wayfinding, you are nuts. Only use this if you like going on half-assed adventures into the unknown.

No.

The bug was then closed as "WONTFIX".

A bunch of people are reacting to this, presumably based on the assumption that no other communication has occurred. I don't know any of the people involved, or even anything about the project, so I can't even begin to speculate as to whether there were other discussions "out of band" or otherwise not shown in this bug.

One thing is clear: if you respond to someone in this manner, you run the risk of generating a rather large and unruly response. I can say this with some certainty based on my own experience. Yep, a couple of years ago, I did this myself. It was in a big corporate environment, not an open source project, and it was a code review, not a bug, but I did use the same response:

No.

I told this tale in one of my older posts back in July 2011. If it seems familiar, you might have encountered it there or in my book. If not, go check it out. It'll make the rest of this make more sense.

There's some more to this story which I didn't tell originally. It has to do with what happened with the actual code review, and it involves some of the finer points of corporate policy about code in the "depot".

My "no." response actually happened on the second code review for this problem. There were actually two of them which I condensed for the purposes of my original story. A few days into this, after he got my original negative response, he went ahead and submitted it anyway. Corporate policy was that you have to have a positive review "score" on your changelist before submission. The tools even enforce this... sometimes.

For whatever reason, he was able to submit it, and I got a mail from the system which said "Changelist #xyz, which you are listed as a reviewer, was submitted. If you did not do this review, please contact blah-blah-compliance ...". I hadn't reviewed it positively, and it had been submitted, so I did like a good little drone and reported it.

I also put together a CL to roll back this change and bounced it off someone else to get the "ps | ... | xargs kill -9" thing out of my tree.

He said it was an accident. Maybe he did in fact submit the wrong changelist number. I didn't even think it was possible to submit something with a negative score unless you used the "to be reviewed" flag to force it. The so-called "tbr" switch was there so you could make an operational change when a reviewer was unavailable but something really needed to be added.

It was appropriate for my former life as a pager monkey, but it made no sense for something like this. Either he used that switch, or he found some kind of problem with the review framework which allowed the submission. Either way, I was incensed. He submitted something which I had explicitly rejected.

Once it had been reverted, we were back to square one. It was this second code review which sat idle for four months, and then when he came back and asked if it was okay now, I said my "no" and left it at that. That, of course, brought on the wrath of both his manager and my own, as explained in the original story.

There's one more thing about this story. When I wrote all of that stuff in the code review and detailed how I had gone to the trouble of trying to train him how to do his own job, I did one other thing at the very bottom of that mega-response, and included something along these lines (bear with me, it's been about four years, so the wording isn't as precise as my "no"):

And finally, I am giving this code review a positive score so it can be submitted. I don't want to argue about this any more. I want it out of my face, even if it is the wrong way to do things.

Just be aware that if this test starts causing problems, I will not hesitate to disable it in the testing system, and I *will* file a bug with you to get it resolved properly.

LGTM.

Yep. I actually gave it a positive score (+1) and walked away from it. It had been nothing but drama and I wanted it to go away.

Oh, and this guy? BS, MS, Ph.D., Comp Sci. If you ever wanted proof that computer science is not software engineering, here it is.

It seems a number of people have decided to start running their own hosted reader on something they control, like a dedicated server or perhaps just a virtualized instance somewhere. There are lots of new log entries where the referrer suggests a web server virtual host set up to do nothing but handle RSS and Atom. The way I see it, that's a fantastic thing. Distributing these services is a great way to ensure they live for a long time and can't be co-opted by some company which doesn't have your best interests in mind.

Still, I wonder about those folks who don't have and don't want to run a full-blown server. They should still have the option of having control over their services. I think there might be an option which I had overlooked for the longest time: the humble consumer-grade router. I recently had to replace mine, and the experience has opened my eyes.

A couple of weeks ago, my connection to the outside started flipping out. At first I thought it might have been my ISP, but then it seemed to have odd timing. It would mostly dump all of my outgoing connections from other machines when I woke up my laptop and it started connecting to the outside world. I figured it was maxing out some kind of connection tracking table in my dumb little router box and the rest followed from that.

At the time, I was running a boring little Linksys box and knew little could be done about any problems with it. It would need to be replaced. Some digging turned up a new vendor which ships ordinary residential type routers with a real Linux environment called DD-WRT. It had been designed to allow you to enable all sorts of useful things. I ran out and bought one before Frys closed one night, and my dropped connections disappeared.

This got me thinking about using this little router box for other purposes. It can see the outside world, and as a result, other things can see it. I could easily tunnel back to it, for instance. The more interesting thing is the possibility of running additional services. They'd have to be small and solid to fit in that resource-constrained environment, but that's why we have languages like C and C++!

While I don't personally need to host anything on my router since I have a real dedicated server out in the world, it seems like this could be an interesting opportunity for others who don't have that option.

Maybe there will be a renaissance of the personal BBS using devices like this. Imagine it: a device which keeps your favorite cat pictures and messaging with friends running even when your "real computer" is offline or not even at your house. It's a device you already have and already keep running continuously. It's just a matter of making it do a little more work.

I'll know this is finally a "thing" when I can walk into a store and buy a device which already has these features preinstalled. If that should ever happen, those behind "cloud" services might need to reconsider their business models.

What does a list of movies look like? It's probably the sort of thing you could make with an unordered list in HTML. They'd probably need to link somewhere useful, but this is nothing the web can't do. It seems like pretty simple stuff, right?

Well, if you're on iOS 5, it will use a request for "which movies are opening this weekend" as a way to use guilt in order to make you "upgrade". It clearly knows what you're talking about, but for whatever reason, is not going to fulfill your request. I'm sure they have some good justification for this, naturally.

All this just for a stupid list.

At a company where I had a three-digit employee number, once I got off the "feed lot" support floor and into more of a development role, things got interesting. I had an actual manager who would knew the sorts of things I had done, and had some idea of what I could do. He'd go off and talk to people on the floor or elsewhere in the company and would return with a project.

It was basically myself and a friend who had also escaped from the larger world of support who directly reported to this one person. Collectively, we were rolled up in a bigger meta-support team which had eight people total. We all had the same administrative boss, but there was an extra "firewall" type person between my little sub-group and the rest of the world.

A bunch of stuff happened this way. In the short time this scheme existed, we completed a whole bunch of interesting tools. There was also my "fallback project" which got cycles any time other things weren't going on. This was my little pet idea of putting a "helper agent" on customer servers to handle repetitive tasks automatically. When people weren't asking me to run reports or come up with some new way to audit the customer service world, I'd work on my agent project.

The problem is that this boss wasn't especially loved by some of the people with more political power in that company. He had a way of telling them the truth instead of what they wanted to hear, and they didn't like that. People who spent time blowing smoke actually went places around there, and he wasn't one of them.

Maybe four or five months into my tenure on this new team, he was "sent back to support". He had to join one of the support teams and go back to taking calls and working tickets. Instead of building the future, he was forced to stay and mop up the present. Not surprisingly, this didn't last long, and he quit.

My friend and I continued on as our little subgroup, and had another person from the larger group assigned as our new firewall. This guy probably meant well but he was no replacement for the original. While this new one had a reputation for being some kind of technical ninja at the company, that was probably just because the bar had been set so low historically. We basically humored him and kept working on the existing projects which our previous boss had queued up.

Eventually we cleared out the backlog, and in theory that meant my agent project would have more time to go places. Indeed, my friend started working on it as well, and we tried making a port of it to Windows using Cygwin. We actually managed to get a fair bit of this code to run and do useful work. At one point, we had it graphing disk utilization and other banal things like that just to show that yes, it is possible to create this sort of thing.

Nobody really cared, though. Some people really wanted to plow money into things like BMC PatrolExpress and other products which managed to be even worse than that. Many of these products had no business being used in a web hosting environment where every box may belong to a different customer. They usually also didn't play nicely with the ground truth that most techs, even the ones supporting Windows, tended to run Linux. Monitoring systems which demanded ActiveX controls forced IE, and that forced Windows. It was incredibly stupid.

My friend wound up finding another job, and he left. Then the rest of my larger team disintegrated as our admin boss followed her boss the VP downstairs to run the "big expensive customer" division. I also coincidentally got kicked out of my cube with higher-than-normal walls at the same time.

In the space of a month or so, my former boss (and friend) who had been kicked out to support quit. Then my friend and coworker who had been working on projects with me quit. Then my team imploded when our boss moved divisions, and I lost my nonstandard cube. I wound up back in a normal support cube working second shift tech support yet again.

It was fortunate that I had started the ball rolling on finding another gig about six weeks before this turn of events. By the time those guys quit, I had already flown out to the west coast to interview in person. Once I got back, it was just a matter of waiting to hear back from them. This was one of the longest waits of my life.

I mean, I had interviewed here in the valley on the 21st of one month, and they finally sent me an offer on the 14th of the next. In the middle, I just had to hold on and wait to see what happened. During that time, rumors were swirling around. Everyone at this place basically knew I was probably the next to go.

Everything was changing around me, and while I had a stable place to "land" back in support, it wasn't a fun place to be any more. Many of the people who made it what it was had quit, and I had been put back in the trenches.

Is it any surprise that I went for that offer right away?

Well, those bad old days didn't really leave. They just changed form.

I don't even remember what brought me to this web site at such an early hour, but once I landed there, I knew it was a disaster. Let's just measure this mess vertically to quantify how screwed up it is.

The image itself is 940 pixels wide and 640 pixels tall - standard fare for my iPhone 4S flipped on its side. I don't remember why I was browsing that way. I just was, and they should handle that gracefully.

The top 40 pixels are given to iOS. They aren't available. From 41 to 150, it's nothing but their obnoxious bar trying to get me to install something when I just wanted to look for some info on their web site. I usually try to scroll past them since clicking the X is usually a disaster which ends up taking me somewhere else when I miss. This one didn't scroll out of the way.

Lines 151 to 238 are given over to yet another banner to remind you which web site you're on that still hasn't given you any content. It also has the usual mystery-meat navigation.

From 239 to 348, there's an unrequested search bar (I didn't ask to search for anything) which can't be closed. It's permanently in the way.

Then from 349 to 458, they want my zip code, and again, they won't get out of my way. I just wanted to browse around, and none of this stuff I have encountered so far is going to let me do that. As for my zip code? I'll tell you that if I have a reason to, and right now, I certainly do not.

Finally, at line 459, it starts rendering graphics and other stuff related to actual home improvement products -- you know, the stuff I was probably looking for in the first place?

Of course, that fizzles out all too quickly, since line 575 is a border, and the rest from there down to 639 is owned by the OS and off-limits for the web site.

So let's review. My display is 640 pixels tall. The top 40 and bottom 65 are used by the OS, so knock out 105, and now there are 535 rows for the web site to use. Of those rows, only 21% of them are given over to the stuff I actually expected to see. Everything else is something which is just in my way.

I should note that only that little spot at the bottom is actually scrollable. Trying to scroll the rest of the page is an exercise in futility since all of those bars aren't going anywhere.

What was I looking for on that site? I can't even remember now. One thing's obvious, though: I never found it. When confronted with this, I go somewhere else.

Why are these worse than the days of browser bar wars? That's easy. You can remove those addins from your browser one way or another. A site like this, however, is not negotiable. You take it that way or you leave it. It's not like you can say "disable style sheet" on Mobile Safari, after all.

Maybe it's a bad design. Maybe I'm just cranky. Maybe it's both.

In Texas, you just don't see that nearly as often. Odds are, there will be a way to turn left directly into the place you're going. This might mean a center turn lane, or something else entirely. I can't remember swinging a U-turn at a light in Texas, whereas it's basically mandatory for certain destinations around here.

That's why this is doubly perplexing for me:

I realize this is a still image, but trust me, that van was in fact in motion. It isn't someone who just decided to park in a very strange place. They did something that would be odd in that environment all by itself and then decided to do it in a place where it was explicitly forbidden. Genius!

...

Do superfluous apostrophes bug you? I die a little inside when I see them, and it's usually related to how big, how visible, or how permanent the error happens to be.

With that in mind, check out this snapshot that I managed to get somehow:

Florida Moving System's, Inc: all caps and on a giant rolling billboard. I bet they have a whole fleet of those things. How does this happen?

...

Today's final example of road strangeness comes courtesy of this sign on the Milpitas/San Jose border. It's where San Jose's Old Oakland Road changes names to Main Street. From what I understand, it used to be the way to go before the freeways went in.

The problem is that in abbreviating it down to just "Oakland", the sign creator managed to set up a potentially confusing situation. Oakland, as in the big city across from San Francisco, is in fact way off to your left when you're looking at this sign. If you turn right to get there, you'll get lost.

I'm going to be charitable and assume this had something to do with a minimum font size and a maximum sign size. It's just odd.

It took a non-trivial amount of time to get this stuff working. None of it came anywhere close to being usable when I originally got my hands on it. I ran into all sorts of crazy problems with this system which I had never touched before, and had to plow all kinds of time, energy and money into making the system which exists today. It was hellish.

Two weeks disappeared into trying to make this thing log audio: it arrived around 11 AM on July 15th and I didn't have a viable demo web page with real recordings until July 29th. It took even more work to make it into the web application it is today.

Given all of this, how should I react when people show up at my web site and then get annoyed because there's no source code to download?

There are folks who get in touch with me because they want some custom radio work. We talk about what needs to be done, come to an agreement, and then I build something. I receive compensation for my work and everyone is happy.

Maybe there's a solution just waiting to be found here. I suspect the people who just want to download something are not likely to engage my services for systems programming. However, I bet they know people who need that sort of work done, and could help with some introductions. If those introductions turned into some gigs, it might be enough to fund the release of some of these projects.

Thoughts? Objections? Alternatives? Let me know.

It took a while to do this. I had to figure out exactly how the publishing process worked from end to end. Ultimately, I had to write a new "renderer" to take the existing flat text files for each of these posts. It's driven by a config file which either contains settings like the title or points to filenames with content. That's how it finds the "intro" at the beginning and the "about" at the end.

This renderer is also responsible for reading a list of sections, each with a title and a list of posts. Here's an extract from the first section:

section {
  title: "User Interfaces and User Experiences (UI and UX)"
  post_id: "2011/05/08/visions"
  post_id: "2011/05/23/userswant"
  post_id: "2011/06/14/xmencoding"

This list then forms the table of contents at the very beginning, and sets the stage for the work loop which actually glues in all of the posts. Each post is parsed into HTML, much like when the web pages or Atom feed (or protofeed!) is built, but with a few differences: all links to other posts inside the book are changed to "#" type links, and some non-ebook-compatible things are stripped out or otherwise reformatted.

Before each post, I have it emit an anchor with a name attribute. Then, any post which does a link to another can just point at that anchor. In the event the target post is not in that book for whatever reason, it will just generate a link back to here. That means if you're on your Kindle (or whatever) and see a link to a post, you can just click on it. If it's in the book, you wind up somewhere else in the book. If not, it'll kick off your web browser and bring you out here.

This year, I'm back at it. This time, it's all of the posts from 2012 which are on the table, and it's a far bigger job. Whereas "The Bozo Loop" was built from about 250 posts, this new one is up over 450 at the moment. It makes sense given that 2011 was a partial year of writing which only began when I quit (in May), and 2012 was all the way through.

In response to some of the reviews, I've set things up to include more "glue" to tie all of the posts together. It's new content which doesn't show up anywhere else. There is also a distinct effort to explain who I am, what I do, and why I do it as a response to another review.

At the end of the day, it might still be seen as a "collection of blog posts", but I hope the quality and depth of my writing will make up for the fact that I publish things online before cutting them loose to the whole world in a book.

I mean, I could just do all of this stuff in seclusion, offline, and then release a compilation of everything now and then as a book, but would that really be an improvement?

The new book is not quite ready yet. I still have to finish writing more new material for each section, and then there is at least one more proofreading pass to handle. There's also the matter of coming up with a cover -- this time I don't have an iPad, so it shouldn't look anything like the first book's "Charlie Brown color scheme" cover.

Rest assured I will let everyone know once I get this thing done. I'm looking forward to having the cycles back to make more /edu/ recordings and generally experiment with some new ideas.

If you have some great idea regarding something I should do in book #2, now is the time to send me a note.

It's a lot of work but I hope everyone will enjoy it.

Hang in there. I'll have another update soon.

Here's a challenge to anyone of the sysadmin persuasion: read through this without skipping ahead and see when you realize what's actually being requested here. Then see how many people totally miss it.

OK? Here we go.

Did you see that "Tech5" basically wanted to change the A record for the actual hostname of the customer's server at the hosting company?

Did you notice that 6 different techs were involved, and that it started on Friday at 7:25 PM but the customer wasn't satisfied until Wednesday at 6:24 PM?

Go back and look at what happens around the time of those internal comments on Friday night. The ticket sits there with no response to the customer's Friday/8:16 PM comment until Sunday at 9:40 AM!

This ticket should have taken at most 10 minutes to do. That's 5 minutes to flip through their ticket history to see which domains they touched last time, 3 minutes to twiddle the SPF records in the DNS tool, and 2 minutes to send an update to the customer.

Absolute insanity.

There are a bunch of things to notice here. First, there's a big red sign demanding that you not smoke in this area. It seems like people might smoke here because, well, there's a bus stop right here. The bus bench and VTA sign make this pretty obvious.

No smoking at a bus stop? Is it something special about bus stops? Actually, no, it isn't. Look just behind that fence.

See all of those pipes? One of them has a yellow label which says "FUEL GAS". Oh, my, what's this, then?

Well, it turns out that it's the gas compressor building for the power plant across the street. It takes natural gas from PG&E and sends it under the street to run the turbines. There's a lot of potentially flammable and/or explosive stuff happening in there, mere feet from the bus stop.

Is this really a good idea? I guess it's good that they have a fire hydrant right there. That way, if something really bad happens, maybe the resulting explosion will blow the hydrant off. If you've ever seen the plume which comes from a sheared-off hydrant, you know a lot of water shows up in a short amount of time.

I took that picture in 2010. Since then, that bus stop has been changed a little. The bench is gone now.

Do people smoke less when standing?

Other people and companies come along and build their own tools. When they do this, they try to compete on function completeness. Since the original insane tool had 1000 functions, they try to approach it as well. It doesn't matter that some of these functions are silly, others are ugly, and some are just plain annoying. More than a couple of them are flat out dangerous in the wrong hands, and some of them have no sense existing in the modern world.

Still, they try to compete on those terms. Maybe one company gets up to 500 and calls it pretty good. Then another manages to hit 600. The one-upping continues every year, with everyone trying to climb closer to fitting all of these features into tools of their own design. The whole thing is ridiculous and pointless, because nobody really needs or wants all of that junk.

This is a lot like the world of programming languages today. Lots of things which are possible still aren't a good idea, and yet they persist. Everyone has a different rationalization for why some of this stuff still exists and is enabled by default. Sometimes I wonder if some of these people are incapable of thinking in such a way that lets them realize that sometimes, fewer features is the better way to go.

For example, if you're a C programmer, when was the last time you used gets() or any of the str* functions which don't take length arguments? If you've actually used them any time in the past 15 years, did you have a solid reason for it, or was it an accident? Were you perhaps a newcomer to the field and didn't know why some of these functions have been deemed dangerous in many situations?

Even if you have a bona fide use case for some of this tricky stuff, do you really need it enabled globally and by default? I bet you could actually live with it disabled unless explicitly switched on for a specific module. Then you could use your dangerous tools in the proper controlled environment without endangering the rest of your project.

There's a lot of crazy, silly, and dangerous stuff in programming languages. I use C++ a lot and so I wind up selecting those parts which are a bit more sensible and leave the weird stuff alone. Just because it's there doesn't mean I have to wrestle it into my code. There's no point in showing off.

Want to see how bad it can get? Okay. Check out some slides from a 2010 presentation called "The Dark Side of C++". Clearly, many dragons lurk in the depths of that spec, but so what? They should be located, called out, and walled off.

It's not like we're a bunch of hunter-gatherers who live off the land and have to respect Mother Nature by using everything we obtain. When it comes to programming languages, you don't have to use the whole buffalo.

The particular flavor I went for was AP Computer Science. I already knew a bit about programming, and was already helping out with day to day operations of my school's Unix box, so why not? It also seemed like the logical progression from the "Computer Programming 1/2" classes I had taken the year before.

I signed up for the class. It was rather loosely run. On one side of the classroom, maybe 20 or so students did the AP Comp Sci curriculum. On the other side, at the same time, another similarly-sized group of kids were doing "Computer Applications" - they were using WordPerfect and things like that. Somehow, one teacher ran the whole thing at the same time.

Not much really happened in there. We'd be pointed at our latest assignment, and then we would go off and work on it. There were sufficient ambiguities to where it was possible to just sort of get something working and it wouldn't really matter how it happened. One time, I wound up "inventing" (not really) my own pathetic "sort" because the assignment required a sorted output.

Want to know how bad it was? Okay, here, I'll spill the beans. I don't remember all of it, but I do know that it involved actually doing "inserts" and "deletes" on the array. Yes, an array. Not a linked list. When I say insert, I mean in the sense where you have "abcd", and you put your cursor on "c" and hit your - key, and now you have "ab-cd".

To actually do this, I would start below the insertion point and I would just copy everything down a spot. Then I'd stick in the new value at the insertion point. Then I'd go do a deletion for the old position, and that involved similar stupid copy gymnastics: start there and copy everything up a space.

This actually worked because the stuff we were doing wasn't very complicated and nobody really cared about speed, but looking back, it's supremely horrible. Nobody noticed. Nobody tried to set me straight. It was basically rubber stamped and I was sent to work on the next assignment.

I actually didn't start writing this post to talk about the useless assignments which didn't really help me at all because I was allowed to flail around without any sense of feedback from the teacher. Oh no. This is about something which failed in a much bigger sense. It's about the actual AP exam at the end of the year.

Here's how the whole thing happened. The tests weren't held at my high school. We all had to find our own way to this special site which actually wasn't a school at all. It was some auxiliary building of the school district which had been a school many years before but now housed something else. I had a license by this point in my life, but didn't have a car, so I borrowed my dad's for the day. One of the students in my class needed a ride so I helped him out, too.

After our first class, we hopped in the car and set out for the testing center. Even though it was mid-May, the weather didn't care, and we had sleet coming down. This is the thick stuff which actually sticks to your windshield and needs to be wiped off. I turned on the wipers.

About halfway between my school and the freeway entrance, the wipers quit -- they actually stopped dead-center on the windshield. The controls did nothing. Off, low, high, intermittent, whatever. They wouldn't move no matter what I did. I was going maybe 35 mph when this happened, and within a couple of seconds had zero visibility. Fortunately, nobody was around, and I knew what the road was like from experience, so I just pulled off to assess things.

While on the shoulder I tried turning the car off and on (hey, it worked for Windows machines) and a bunch of stupid things like that. Initially, nothing happened. Then, oddly, they came back to life. I decided to get moving before something bad happened, and we took off again.

A minute or two later, I was on the freeway for a short hop - it was perhaps a mile to the next exit, and that would lead us to the testing place. What happened? Yep, they quit. On the freeway. At 55 mph. Once again, I had zero visibility in a matter of seconds. This time I was really in trouble. Pulling over wasn't an option this time. The exit ramp was coming up soon... just... when?

I did something relatively bold and stupid and rolled down the passenger side window, and told my classmate to watch for the white line. When it moved over to the right, that was the beginning of the exit and we were going to take it. Then, out of the flow of relatively fast freeway traffic, we'd figure it out.

He leaned his head out and watched. I just drove straight, trying to see out my own window, all the while fighting with the stupid knob. The exit ramp started, he called it out, and we exited. Just then, the wipers came back on at full speed, making a terrible racket since there wasn't enough moisture to keep them busy. I didn't care. I wasn't touching that selector again. Instead, I took off at an expedited pace for the testing center, some two miles up the road.

Somehow we got there without running any red lights or hitting anything. We were actually early and had to wait around for our session to open. It was held in a big open room with a bunch of normal classroom desks. There were no computers present.

In the actual test, the first part was a bunch of questions in a booklet. I don't remember much about it, and it's not important anyway. Where things turned into a disaster was part two.

When they turned us loose on the second half of the exam, I opened the booklet to discover a ton of code for some kind of "directory manager" program. I had never seen it before. I guessed based on that name and a quick scan of the source that it was something like Norton or PC Tools or whatever: it probably gave you a list of stuff, and you could rename things, or copy, or move, and things like that.

The test asked me to add some kind of feature to the program. I don't remember the finer points of it, but this is how I interpreted the situation, and about what my inner monologue sounded like:

Okay, so they want me to read through all of this and understand it, effectively compiling it and running it in my head, since I've never seen this before. I have to do this to the point of understanding where everything is, and how it works, and how it might be extended. Then I have to figure out how to extend it, and then actually extend it. Finally, I get to write this code out with a pencil, since, again, no computers.

I just sort of looked at the test pamphlet and pondered my options. I couldn't exactly get up and leave. They wouldn't let me disturb the environment by getting up, and besides, I had to give my friend a ride home after the test.

I could try to actually do it, but that would be a serious amount of work, and for what? Just so I can possibly get a 4 or 5 on the exam and maybe turn that into class credit in some college I haven't even selected yet? What if I do all the work and don't even get a 4? Then it's all for nothing.

The inner monologue continued.

This is a ton of work: all on paper, no computer, not even a compiler to make sure it'll parse and actually run, never mind run correctly. If they're going to demand this much work from me in a stupidly short amount of time, they should be paying *me* for the privilege, not the other way around!

I decided to not do it. I still couldn't get up and leave, and I'm sure there would be some kind of anomaly if I didn't write anything, so I picked up my pencil and wrote something really stupid to establish that yes, I was physically present and just didn't do it for some reason. The "code" I wrote looked approximately like this:

foo := bar;

That done, I put down my pencil, closed my booklet, and just sat there and stewed until time ran out and they let us go. I took my classmate home and then went home myself.

A couple of days later, I was told what had happened to me. Supposedly, we should have encountered the "Directory Manager" thing as part of our curriculum. That is, we should have been working on it all year to get familiar with it, and extend it in other ways. That way, when we encountered it again on the exam, it wouldn't have been a surprise.

[ I should note that I had no way to verify this, so this may have been a yarn someone spun to "explain it away" for me. I can't prove this is how things were supposed to work. It's just what I was told. It's also possible I took the wrong test somehow (A? AB?) but why would my teacher let that happen? ]

The source code they provided in written form was just there as a reference. We should have known our way around it from the past 9 months of hacking away at weekly assignments on the thing. Instead, we just putzed around in this textbook doing a bunch of stupid assignments that didn't really get graded anyway. Oh, they got a score, but they weren't sanity-checked. The "bogo sort" flavor I created proves it.

When my score arrived, it was uselessly low as I expected. The whole thing had been a waste of time and energy. I should have just skipped it and gone to my usual classes that day instead.

What happened here is simple enough, and it's a pattern which has reappeared in my life: when faced with sufficient stupidity, I decide to walk away from it. Sure, in theory, I could have tried to do some kind of heroic "save" to figure it out on the spot. In practice, who the hell cares? I may have been an ignorant teenager but I still had a bogon detector, and it was pinging wildly at the events of that day.

Instead of stressing out, I just wrote the whole thing off.

The actual impact on my life seems to have been zero in the long run. Nobody cared about that test when I applied to schools a year later.

...

After writing the above, I went digging through my old floppy images... and found the infernal 'sort' code. Ready to be sick? Okay. Here are the innermost parts which did the really nasty push/pull stuff:

PROCEDURE PushDown (Var ToPush: ListArray; PushFrom: BYTE);
VAR
  PushIdx: BYTE;
 
BEGIN
  FOR PushIdx := 100 DOWNTO PushFrom DO
    ToPush [PushIdx + 1] := ToPush [PushIdx];
  ToPush [PushFrom] := ' ';
END;
 
 PROCEDURE PullUp (Var ToPull: ListArray; PullTo: BYTE);
VAR
  PullIdx: BYTE;

BEGIN
  FOR PullIdx := PullTo TO 100 DO
    ToPull [PullIdx] := ToPull [PullIdx + 1];
  ToPull [101] := CHR (255);
END;
 
PROCEDURE SortArray (Var ToSort: ListArray; ListPos: BYTE);
 
  VAR
    SortIdx: BYTE;
    CurrCh: CHAR;
    NextCh: CHAR;
    NextCh2: CHAR;
 
  BEGIN
    ToSort [0] := 'A';
    ToSort [101] := 'Z';
    CurrCh := ToSort [ListPos];
    FOR SortIdx := 0 TO 100 DO
     IF ListPos <> SortIdx + 1 THEN
      BEGIN
        NextCh := ToSort [SortIdx];
        NextCh2 := ToSort [SortIdx + 1];

        IF ((CurrCh >= NextCh) AND (CurrCh <= NextCH2)) THEN
          BEGIN
          PushDown (ToSort, SortIdx + 1);
          ToSort [SortIdx + 1] := CurrCh;
          IF (SortIdx < ListPos) then
            PullUp (ToSort, ListPos + 1)
          ELSE
            PullUp (ToSort, ListPos);
          Sound (500);
          Delay (1);
          NoSound;
          Exit;
          END;
      END;
 END;

See the "Sound(500)"? This thing actually ticked every time it did a pass because it took a while to run. This way you could be sure the machine hadn't locked up. I had forgotten about that.

All that to sort 100 random characters. What a miserable mess.

Officially, I did them when working for that place in Mountain View between 2006 and 2011. They had given the practice an actual name, and while I doubt they invented it, it was widespread inside that company. The way it worked initially was simple enough. Every Monday morning, you'd get a nag mail from the system. You had until some time that evening to respond. If you replied before the cut-off, it would automatically populate the previous week's entry with the contents of your mail.

I used to keep a flat text file in my home directory and would prepend things as I worked on them. The file grew upwards, in other words, so I didn't have to keep scrolling to find the right place to add something. Then, on Monday morning, I'd encounter the nag mail during my usual mail handling pass, and I'd paste the latest chunk in. It would be happy and that would be that.

What did a typical snippet entry look like? That's easy. It's stuff like this:

- AM oncall
- pushed Foozle package v.123 to nebraska-west
- chased down inter-cluster connectivity bug in IX (b/12345)
- hooked metabot up to production logger

This is just an example, since I don't actually have any of those. They're all locked up in some corporate system which might not even exist any more.

Those four lines documented a lot about what I actually did in that week. I wore a pager and dealt with whatever happened as a result of that. I did a push of some new software, dealt with bug #12345 pertaining to some connectivity issues, and added a feature to an existing program called metabot.

Thinking back, I actually did this before I got to that job. At the dev gig I had before that, someone got wind of what this "magical place in MV" was having its engineers do. They heard about the weekly writeups but missed the part about them being *brief*. As a result, I started writing summaries of what I did, and they were much longer than the above. They were written more like e-mails or some of these posts here, and did not resemble a bunch of bullet points.

Looking back even before that, I found a bunch of old entries which could plausibly be called snippets from the '90s. Yep, back when I was running a flock of machines for a school district, I was keeping notes of what had happened. This actually happened for a couple of years before I got the idea to make a coherent sysadmin notebook for all of my machines. Many of those older entries were used to "backfill" the notebook for systems which had been around for a while, in fact.

What did a "snippet" from back then look like? That's easy.

- RBL measures installed on mx2 (sendmail x.y.z)
- nightly log rotations configured
- sendmail x.y.z on mx1
- added virtual domain mail handling to mx1 for dom2 and dom3
- linux migration work:
- turns out bsd/os shadow passwords work fine on linux
- conclusion: mass password change unnecessary
- mail change: envelope froms must now be fully qualified
- upgraded qpopper to x.y
- sendmail x.y.z on mx3

Yep, the "RBL", as in the original DNS black hole list. Like I said, it was the '90s. This is the slightly-sanitized entry for just one day of sysadmin monkeying I did one summer long ago.

All of this brought together makes one thing obvious: I have always liked to write about the things I'm up to, even if there's no guarantee of it ever being read. If nothing else, it's a terrific way to answer questions like "when did X happen?" or "what were you doing on Y?".

...

A couple of years into my run of doing snippets for That Place, I had noticed a few things. Some people didn't write anything about what they did. Their managers obviously didn't care, and so on up the line. When it came time to figure out what they had been doing for the purposes of a peer review, you couldn't use that as a hint.

I never figured out what made people ignore these things. Were they "too good" to explicitly tell people what they were up to? Did they do it on purpose so you couldn't see how little they got done in a typical week? Was there some other devious plan? Who knows.

...

Then there was the time they started screwing around with the way it worked. First, they moved the nag mails to Friday afternoon. Now you had to make sure you didn't whack that mail if you encountered it before Monday morning when you'd normally do your update. I think that only lasted a week, though, and it apparently reverted to Monday.

Not too long after that, they disabled recognition of holidays because, well, "Mountain View is not the world", or something like that. Instead of localizing the system for the user and their set of holidays, it would just pretend none of them existed.

You see, previously, it knew about Mondays which were holidays, and would delay the nag mail until Tuesday. This was great, since you could then reply to it by that magic time on Tuesday and it would still apply to the prior week. It assumed that you didn't log in to work and do mail on the holiday, and gave you an extra day to sync up. It worked just fine in my experience.

When they turned off holiday recognition, it started nagging everyone on Monday no matter what. Of course, this happened around Labor Day, which is a Monday holiday in this part of the world. By the time everyone got to work on Tuesday, it was too late to just reply and have it go to the right place. If you cared about getting it right, you'd have to manually go to the site and drop your update into the prior week.

I always wondered how many people had their snippets shifted due to this. If you think about it, it would make some weeks have no report, and some weeks would get a double dose: the one which showed up "late", plus the next one which showed up on time a week later. It could be rather confusing.

...

Of course, some projects decided they weren't going to write snippets on purpose. One in particular went off and did their whole thing in secret, and locked everyone out of everything they could control. You couldn't discover what they were up to from snippets, and you couldn't see anything in their code reviews, either. The actual code itself was locked down and secret, and you had to be on this stupid list of "special people" to get around any of that.

They missed locking down a few things, so it became possible to see some of what they were up to just by digging around in a couple of unconventional places. What I found didn't seem to make much sense. It was a big bunch of crazy. When it finally launched, it was slow, confusing, and didn't have much happening in there. Hardly anyone knew what to do with it. They apparently didn't know what to do with it, either.

I wonder if the lack of openness make this happen. By operating in secret like that, any sort of oversight from people who had useful experience in different arenas was forcibly rejected. All of those "what are they doing about X" questions did finally happen all at once when they did their internal demo, but it was too late by then. They launched a couple of weeks later, and all of those problems were still there, naturally. There was no way to address them by this point.

Then again, just flaunting the "we're special and you're not" probably earned them the "evil eye" from more than a few people. If curses really exist, I'm sure more than a few were floated around due to this.

...

What project do you think I'm talking about? Do you have some idea?

Are you sure?

Do you realize I could be describing several?

Oh yeah. Believe it.

Writing about that youth challenge program which was held in the old Army jail got me thinking about the summer we had to replace all of the monitors. We had something like 2,000 of these Optiquest 14" monitors all over the school district. We had standardized on them during the rollout of the big multi-million-dollar technology bond which put computers in all of the classrooms, built a bunch of labs, and ultimately paid for my Unix boxes.

I don't remember exactly what happened, but somehow it was determined that everyone of these stupid things had to be replaced. The company was good enough to send an 18 wheeler full of replacement monitors to us, and we imposed on the warehouse guys by filling it with our hardware for a while.

Then we had the problem of actually delivering them and fetching the old ones. We tackled it one school at a time. In the days before we'd get there, all of the monitors in the building would be rounded up and placed in a common location, like the gym, cafeteria, or a computer lab. It was summertime, so they could leave most of them disconnected for a couple of days and not really notice. Then, we'd fill our vans, cars, and a big scary old "stake truck" with monitors and would drive out to the site. We also took along a bunch of cardboard boxes, styrofoam packing shells, and tape guns.

Once at a school, we first had to bring in all of the empty boxes and shells. The company had sent us enough stuff to pack up all of these broken monitors so they could be shipped back. We had to take this stuff into the schools and then box up all of the monitors. This meant finding a "top" piece, a "bottom" piece, then building a box and stuffing it in there. Then we taped it up, stacked 'em 3-high, and wheeled them out on a hand truck.

While this was going on, other people were wheeling in the replacements. We'd keep track of how many were pulled and how many were replaced to make sure we didn't short them on their equipment.

Back at the warehouse, then we had to take these freshly crated monitors and "palletize" them for shipping. There were a whole bunch of those wooden pallets from when the new monitors had arrived, and we had to stack these boxes on those things. I think we'd get 4 on a level, and then either three or four boxes tall. Finally, it ended with someone grabbing a huge roll of clingy plastic and walking around and around to wrap it. I think (and hope...) there were also straps involved, but I don't remember for sure.

This was a huge job, and it was too big for just those of us in the department to do, so we brought on a bunch of student helpers. These were kids who were in one of our high schools and were out on summer break. This was a relatively tough side of town, and the kids knew more about certain sides of life than others, to put it nicely.

That's where the "jail" part of my memory comes in. One of our elementary schools was less than a year old when all of this happened. It looked really clean and new inside and out. It had been built to be not just an elementary school, but also a community center for this new neighborhood which had been built up around it at the same time. The whole thing just looked "institutional"... maybe even more than a school does normally.

When we arrived out there with our student helpers, one of them took one look at the place (since it didn't exist when they were kids) and said to the others "lockdown...", and at that point I got it. It really did look like some kind of jail from the outside. It didn't have the barbed wire and tall fences, but the actual exterior design of that place was cold and imposing.

My experience with the "last chance" school came a couple of years later, and then I really understood what they were talking about.

What I'm trying to figure out is how a bunch of 16 year olds would come to have a reference point for that sort of thing. Maybe they'd been through juvenile hall already? Who knows.

I don't have any pictures of that place, so I'm going to borrow one from Street View in the name of fair use. Here's a view of the back corner of the building where we parked on that day so many years ago, and this is what the kids were reacting to.

Not exactly friendly, is it?

What he wanted was for this program to name its output files based on how many "parts" were in a set. That is, if there were 1-9 parts, the files should be named part1, part2, part3, and so on up to part9. However, once it reached 10, all of the names should include the new position, and thus needed to be part01, part02, part03, ... up to part99. If it happened to have 100 to 999 parts, it would need three positions, and so on.

I forget how this wound up on my radar, but it seemed like a reasonable request. Filenames with digits tend to sort oddly. This is where you'll see "part19", then "part2", and then "part21" in a directory list. That leading zero would keep things in line. I decided to take a look.

One possibility which seemed obvious was to just take the biggest number and format it as a string with sprintf() or similar. Then I could just use strlen() to figure how long it was, and that's how wide the numbers would need to be. I could then build a format string like "%04s" to zero-pad things appropriately when formatting the filenames later on.

Still, taking a number, making it into a string, and then looking at the length of that string seemed wrong. I thought about it for a while and tried to come up with something which didn't do all of that string manipulation stuff.

While talking about this with my friend, I realized I had encountered something which behaved a bit like this way back in high school math. It tended to yield results which effectively resembled the "width" of a number. That function was log. I couldn't remember what the actual point of log was from that class, but knew it might be helpful.

After a bit of poking around, I determined that the function I actually wanted was log10(). It also needed some help in the sense of truncating the fractional bits and then adding 1, but I had my answer. That number was used to create a format string, and then that format string was used while creating the filename.

I still think of that as rather ugly, but it did work. Thinking about it now, I worry if perhaps using that log10() function would fall into the bucket of "being too clever". Clever code tends to trip up people who do maintenance on it much later, including the original author.

It now occurs to me that this sort of thing could be done with a loop which just kept dividing the number by 10 until it ran out of data. The number of passes would tell you how wide it was. That would avoid bringing in a math library just to do something really dumb with a number.

I guess everyone has their pet ways to handle annoying junk like this.

I've learned a few things about making openvpn2 run on a Mac which I figured are worthy of a post to get them out there for the world.

First, if you dig around online, you can find any number of "plist" files for openvpn. The gist is that you create a file in /Library/LaunchDaemons called org.openvpn.plist or net.openvpn.plist, or really anything you want. Then you put a bunch of magic cruft in there, and it will start it up for you.

If you don't want to treat all of this as magic, the good news is that this file format has a man page: launchd.plist(5). If you read that, you will learn something I did not know up front: processes must not call daemon() or equivalent. Oops.

Historically, I've always run OpenVPN on machines where having it "daemonize" itself by forking and detaching from its ttys made sense. However, when I recycled a config file with the "daemon" directive on this Mac, all kinds of stupidity started happening. First, I could watch in the /var/log/system.log and it would keep restarting it over and over. The only thing keeping it from starting tons of copies was the fact that its local port was already bound.

Also, in this state, "launchctl list" would show something like "0x100402c30.anonymous.openvpn" instead of a reasonable entry. It turns out this happens when launchd starts a program and then "loses" it... like if it goes into the background by itself. This is not an easy thing to discover through the available documentation!

If you run into this, just figure out whatever is making your program jump into the background and force it to stay in the foreground instead. launchd really wants to keep its hooks in your program as a direct child, and gets seriously unhappy any other way.

It's also a good idea to take a look at "KeepAlive" and notice that it's possible to just set it to "true" to unconditionally keep things running. If you skim that paragraph and jump down to the four options (SuccessfulExit, NetworkState, PathState, OtherJobEnabled), it's easy to miss the fifth (true).

This seems to be the way of quasi-Unix systems these days. Everyone is reinventing init for one reason or another.

What I haven't described yet is the alternative to that alternative. Yes, they actually had something which went well beyond the carpet-store-turned-school for kids who really had strange things going on. This one was more of a military-style boarding school, and yet I had to get involved with it. Here's what happened.

One fall, we got news that a new school would be opening in our coverage area. Every school district had defined boundaries, and anything which happened in there was automatically our responsibility even if it wasn't directly run by us. In this case, there was a tiny little slice of a nearby Army base which somehow fell into our borders due to some property line oddities.

Normally this wouldn't mean much, but then someone decided to set up a school in that little slice of land. They recycled a building to do it. Apparently, that building was the former Army jail for that base.

So one day, a bunch of us met their staff and synced up with their "technology guy". He had done a decent job of getting things patched together as best he could. They didn't own the building so they couldn't make permanent changes, but he had wires tacked up all over the place and had split things into two separate networks. The staff was on one and the students were on another.

When we showed up, it made his day. We offered to give them all e-mail accounts on our systems (that is, my Unix box) and also dialup accounts should they want to do things from home. We would also build yet another Linux gateway with a modem and would park it out there to get them online for free. They'd have a block of addresses from us and wouldn't have to pay for ISP service any more. The plan was that we'd use it as a stop-gap while figuring out the particulars of getting a T1 installed (through the Army and the telco) and would swing them over.

About a week later, I had all of the stuff built, and drove back out there to install things. While out there for the second time, I got to see more of the facility. It was in fact a former jail, and it looked and felt just as creepy and weird as you might imagine. It was surrounded by a layer or two of tall fences with barbed wire on top. Every door was solid and heavy.

They didn't use any of these locks to keep people in (or out) any more, but it still looked forbidding. I can't imagine how this was a good thing for the kids who wound up going here. Imagine being on the edge of falling into the wrong side of the penal system in the first place, and then spending most of your days in a glorified jail anyway.

This actually managed to run for a full school year of 9 months and they graduated a single class. After that, their funding was cut and they had to pack it in. I went back out to collect my server and said goodbye to everyone.

There was one part of this which really stuck with me. Apparently they'd have kids "bug out" now and then. This is where they decide they aren't doing this any more, and, well, take off running. They actually made a small request of the kids for when they decided to do this, and I got to hear it:

If you run out, could you at least run out the front door? If you go out through the side doors, it sets off the fire alarms, and then we have to deal with the fire department, and that's not cool. Just go out the front. We don't keep any of it locked, so you can just go out and it won't set off the fire alarm. Okay?

When you're running on a thin budget, you don't get to be picky about things like real estate. If someone offers you a building, you're probably going to take it.

As he was telling me, in previous years, there was no way to know when they'd go on sale. You just had to be there when it happened to sign up. Not only was the time of day not known, but the day itself could vary greatly. It sometimes happened months before the event, but started creeping closer and closer to the actual event as time went by.

Last year, apparently it opened at such an early hour that people out here on the left coast were probably still asleep. A bunch of folks set up weird monitoring schemes to keep pinging the signup site which would set off alarms when it went live, but they still weren't always successful.

This year, there was a new rub. It seems they announced the exact time and date when tickets would go on sale, and so the entire world showed up right then. Instead of having it just happen at some seemingly random time, now everyone knew to purposely be awake and online this morning.

What happened is that while my friend got his ticket, this other friend of his who has historically gone with him was unable to get one. Now I'm sure there's a question. Does the one lucky person go alone, or does he cancel because it's no fun anymore? If he goes anyway, will the other person feel left out, or will it be okay in the end? I'm sure lots of people are having to face this dilemma right now.

This whole "race to sign up" thing seems so artificial and unnecessary. They could just as easily have people register their interest days, weeks, or months in advance. It might even be a bit you set on your Apple account (or, well, one of them, that is) and say "please consider me for WWDC". Then, once a year, they have this week when they randomly pick people from that set.

My understanding is that WWDC has enough room for 5,000 people. A week has 10080 minutes in it (assuming no DST transitions or other calendar oddities). That means they could start at midnight on Sunday and pick one person every two minutes for the whole week and would wind up with 5040 candidates.

Imagine how that would play out. Instead of some mad crush in which the entire Apple-loving world beats the hell out of some web site, people would be able to sign up at their leisure. There would be no need to hover over the computer and keep clicking reload. It would either work, or it wouldn't.

What's more, with the spread-out nature of selection, it would become an event unto itself. All of the usual suspects who care about this stuff would get to hop on Twitter and say "I won!". They'd get to bask in the glory for a few minutes or hours until someone else got to post their own good news. There would be "winners" in between, naturally, but not everyone is the sort to brag online.

Not only would the winners get their own little time in the spotlight as the latest lucky person, but other folks who hadn't "won" yet wouldn't feel nearly as bad. After all, even if it's already Thursday, there's still all of Friday and Saturday left! Any of those drawings could be the one in which they are selected.

There's more to this, too. If the selection process was sufficiently capable, then the "I won but my friend didn't" situation could be eliminated. Instead, you'd have an "all for one and one for all". Here's how. Both person 1 and 2 sign up as usual. However, when indicating their desire to go, they both add pointers to each other. That is, 1 says he wants to attend with 2, and 2 says he wants to attend with 1.

Now, instead of being two blocks of one, they are now one block of two. If they are chosen, the selection process just skips a round two minutes later so the number of slots remains the same. I wouldn't take this process beyond pairings. Bigger groups of candidates will just have to pair off and hope for the best.

Granted, this sort of week-long lottery won't whip people into a frenzy like the current system does. Maybe that's what Apple wants. That's pretty screwed up, if so. When you build up those expectations so high, people are bound to feel pretty miserable when they don't get in. That just seems mean.

Also, splitting up groups of friends based on the cold reality of who manages to get online and signed up within 68 seconds is just cruel.

There has to be a better way.

Some places don't lend themselves to stopping the car, and also aren't particularly bike-friendly. For these, you just have to find a distant place to park and hoof it back over to figure out what's going on. This is one of them.

Here in Santa Clara, there is this weird overpass which lets you go more-or-less from El Camino Real over to Coleman. Before it existed, there was no good way to cross over the railroad tracks without dealing with trains blocking the way. There are some odd things happening on either side to make it interface with the residential streets in that area. For one thing, it appears they used to have an extra lane for routing traffic a certain way through here. This is what it looks like at Lewis at Alviso:

Notice, however, they added a curb to keep people from actually using it. Someone who wasn't quite "with it" might see the extra lane and set-back curb and might even try to turn onto it. Of course, if they actually managed to make it over that curb, they'd find themselves going the wrong way on a one-way street. Awesome!

Pull it up on your favorite satellite viewer and check it out. Also note the super tight turn for traffic coming up Alviso which wants to go left instead of right (and over the bridge). The sign which tells trucks to keep out is the only thing keeping traffic on that side from going up that dead lane.

Here's the back view of the "no trucks" sign, and notice there's no curb blocking the way from this end of things:

That whole area has a bunch of weird tight turns and sketchy merges. It feels like a bunch of these structures were shoehorned in after the fact.

...

In other "sketchy roads" news, here's a completely crazy bike crossing for the San Tomas Aquino Trail at Monroe. What we have here is a four lane road (with bike lanes) which is interrupted at the creek for a diagonal bike crossing.

[ Click to see more detail. ]

It's not immediately obvious, but that is a tiny little red bicycle in that traffic signal. It's lined up to point directly at where I was standing in this picture, and there's another one pointing at the other side.

In theory, you push the button, the main road gets a red, you get a green, and you can scoot across on your bike. In practice, however, things get a little dicey. More often than not, I see cars cruise right through here like they're not even aware of a light. It's not even malicious. I've stopped here just to watch the shenanigans on occasion. Most of the people are genuinely surprised by the fact they just ran a red light and, oh, hey, bicycles!

It's not like they're out to collect 100 points, Death Race 2000 style. They're just doing their thing and something about this "intersection" doesn't register as such in their heads, and so they blunder ahead and have close calls with two-wheeled commuters.

My guess is that it has something to do with the extreme skew and the lack of a cross street. That and the absence of actual pedestrians (that is, people on foot, and not on a bike) probably means they just zone out.

Imagine what it must be like. You're driving along, and you see something that doesn't resemble an intersection, since there's no cross street. Also, it's just past one real cross street (Roosevelt, plus the parking lot), and just before another (South/Marmon). This can't possibly be an intersection. Besides, there are bridge rails! There can't be a road here.

All of this is going through your head as you roll down the street, and then, WTF, there's a bicyclist coming at you from an angle, and boy are they angry at you!

I've not been the car driver in this scenario, but I have been the bicyclist a couple of times. The first time it happened, I thought that person was just being evil or stupid. Since it keeps happening, I now realize something is fundamentally wrong with this intersection. It just doesn't get the right behavior out of people who honestly intend no ill will towards anyone.

This is just me beating the same drum again: if too many users are wrong, it's probably your fault.

Here's another look at just how cockeyed this intersection is. They reproduced it nicely on the helpful "how to use this thing" sign.

It's not clear what the correct solution would be here. Normally, this trail goes under the bridges at crossings like this. This probably wasn't an option in this case since there is a confluence with Saratoga Creek just past this point. It may also be too narrow to fit the trail under there without adversely affecting the creek drainage.

Going over the road would be nice, but that would involve a non-trivial amount of money. What they did here was fairly simple: adjust the paint on the road, put in a signalized crosswalk, add a couple of "bicycle signals" that light up with the crosswalk signals, and put up some signs. Actually building a bridge to clear both Monroe and the creek at the same time with compliant grades on either side would probably cost a lot more.

Now, the Stevens Creek trail in Mountain View and Sunnyvale manages to accomplish exactly this. It goes over or under everything it encounters: 101, 237, 85, El Camino, you name it. It took a long time for it to get to that level of coverage, but it happened, and it's pretty amazing. You have to work to get up those slopes, but not having to deal with auto traffic is great.

Maybe this will happen here some day. The city is aware of the situation, and sometimes they park traffic enforcement units out here to keep an eye on things, but that's not really a solution.

First, some background. For my backups, one of the things I use is called a Time Capsule. It's this short plastic box which has a hard drive in it and sits on my network, and gets super hot. It's basically their official solution for "network attached storage". Then, there's some software on my machines called Time Machine, and it's supposed to wake up from time to time and automatically push updates to that server.

When it works, it works. The problem, just like so many other Apple things, is when it breaks. It's obviously unable to recover by itself and requires the kind of manual hand-holding and evil hacks which should not exist in this situation.

The latest failure mode is that it will just stop backing up. There's no warning that anything has gone wrong. The little "circular arrow around a clock" icon on the top bar looks the same. However, if you click on it, you'll soon realize something is up. Instead of saying something like "Last backup: (some time today)", it says "Last backup: April 15, 2013".

Yes, that's what mine is saying right now. It hasn't backed up in over a week, and yet it didn't feel the need to tell me about this. This is for a system which is supposed to run hourly, assuming the machine is on my home network and is turned on (it's a laptop). If you figure 24 hours in a day, and 8 days since it last ran, that's *192 missed backups* with not so much as a peep from this thing. It's inexcusable!

This actually happened to me once before, a couple of months back. The fix is to roll up your sleeves and play sysadmin on a box which claims to take care of everything for you. First, you need to click around and make sure the top level of the Time Capsule is mounted. This usually just means bringing it up in the Finder. Once it shows up as mounted under /Volumes, you're ready for the next step.

Now you have to find the ".sparsebundle" which is associated with the machine in question. It's just the big blob which is the encrypted filesystem. First, it has to be attached... without mounting it.

hdiutil attach -nomount -readwrite foobar.sparsebundle

It should spit out a bunch of /dev/disk* lines in response. One of them should be "Apple_HFSX", and that's the one which now needs to be fscked. Yeah, the same thing which you probably haven't run by hand in years on your Linux box, because those systems have figured themselves out by now. You get to run it by hand. It's going to take forever, and it'll turn your laptop into a nasty little space heater, but that's how it goes.

fsck_hfs -rf /dev/disk(whatever)

This will take hours, even if you have a machine which is otherwise idle and has a gigabit link back to the Time Capsule box. If the machine is busy or has a slower link in between, I imagine it might take days. If you're lucky, it will do whatever needs to be done, and will just dump you back out to the prompt.

If everything went well, then you can use hdiutil to detach the image and then switch your normal backups back on. It might just "grab on" and start backing up. Since it's been stuck for a long time, this means many more hours of having the machine bake itself while it pushes data over to the network disk.

What I can't figure out is why TM apparently won't do this "attach + fsck" thing by itself. It's obviously what gets things working again, and it's supposed to be in charge of that sparsebundle file, so what's the deal? It would be even better if the TC itself did all of it locally to offload the work from my laptop, but that would imply having a disk drive with actual brains. That's been out of style since Commodore went out of business.

Macs don't remove the sysadmin duties from your life. They just hide them and hope you don't notice.

I wonder how much of the relatively slow uptake is due to that relative inaccessibility of the addressing scheme. One might say it's even too flexible. It looks like a "second system effect" to me.

Anyway, as part of these ramblings far in the past, I started thinking about what a scheme might look like on a parallel world. Imagine that IPv6 had never been invented and it wasn't a pressing issue, but people were starting to wonder about the future. Maybe on this world, router RAM was super expensive and so the explosion of BGP routes also forced people to start being creative.

What I imagined was a little like having a switchboard operator for IP addressing. Imagine calling into a big business which only has a single public line with an operator. You call them up, they pick up the phone, and you ask for extension 1234. They connect you through, and that's it. There could be extension 1234s repeated at every big business in the world and they wouldn't conflict because their external addresses were unique.

My weird little idea was sort of like that. It just took the IPv4 space and doubled it. I dubbed it "IPv4++" for no particular reason. Here's how it would work.

I connect to an ISP. They give me a single IP address which is part of their (relatively small) netblock. Let's say it's A.A.A.100. That becomes my "IPv4++" segment. It would look like any other host to the rest of the net. Inside that segment, I have many hosts. How do I tell them apart? That's where it gets a little weird.

The next part involves cramming the "inner" address into the IP header somehow. Maybe it would use some weird new IP options. The point is, it should look like a normal packet to the rest of the world. Existing routers should pass it on just like any other traffic.

Inside my network, I'd give my hosts whatever addresses I wanted. For the sake of simplicity, let's say I use 10.0.0.0/24 and I have two hosts: "alpha" at 10.0.0.1, and "beta" at 10.0.0.2.

Other hosts on the Internet which understand this hypothetical addressing scheme would be able to reach my "alpha" and "beta" machines. Their addresses would be A.A.A.100:10.0.0.1 and A.A.A.100:10.0.0.2, respectively. As you can see, the "outer" address identifies the network, and the "inner" address identifies the host.

There's something else which can be done here, too. Let's say I get a second ISP, and they give me an IP address of B.B.B.200. I use that to feed the same network at the same time as my first connection. What happens now? Well, now all of my hosts are available either way.

That is, you can reach "alpha" as A.A.A.100:10.0.0.1, or B.B.B.200:10.0.0.1. They both refer to the the same machine, but those addresses represent completely different gateways. It's a little like encoding the route into the address.

Imagine a world in which the equivalent of A records can support this sort of addressing. You'd be able to do something like this:

www.example.com.    IN  CNAME  alpha.example.com.
alpha.example.com.  IN    A++  A.A.A.100:10.0.0.1
                          A++  B.B.B.200:10.0.0.1

Any number of things could be done with this information. A client could just pick one at random, much like what happens when multiple A records exist right now. A client could also try to make an educated guess about which address is "better" and purposely select one over the other. They'd have some choice as to how they wanted things to work. Maybe they'd notice "hey, they have connectivity through one of my ISPs" and they'd pick that one for a shorter/faster/cheaper route.

You might notice something else about this: I have just become multihomed by buying ordinary commodity Internet service from two providers. I didn't have to buy a giant router to speak BGP, acquire an ASN, get peering arrangements with the ISPs, acquire a netblock, or anything else of the sort. My single IP address from each ISP is enough to give me a path to the world.

Obviously, this only matters if the hosts initiating a connection actually understand this stuff. Ordinary hosts which speak only the original plain IPv4 would never be able to reach hosts like this. This isn't great, but it's not much different than the situation we have with NAT right now.

Instant multihoming is one interesting thing, but there's more: you don't need anyone's approval to do this if you really want to. If you start running it and someone else does too, then you can speak it between your two sets of systems. Nobody else needs to adjust their network to handle it: not your ISP, not their ISP, and not the providers in between. To all of them, it's just ordinary IPv4 traffic which possibly has a weird option in the headers.

There's another strange feature to all of this: you never need to renumber your actual hosts when changing providers. Sure, your externally-visible addresses will have their gateways change, but you can do that as a "make-before-break" situation, where you turn up provider B before you turn off provider A. The hosts themselves stay on their same internal addresses the whole time.

Right now, changing providers can be an interesting business. If you have your own routable IP block, then okay, I guess it's not too bad. Turn up a new link, get them to advertise you, get the old one to stop being advertised, then turn down the old link. How many companies have enough networking horsepower to do that sort of thing, though?

I saw this happen with my web hosting customers. They'd migrate from one data center to another, and in so doing would completely change IP addresses. The ones who really cared about this would lower the TTLs on their DNS records weeks in advance out of paranoia. Then they'd turn up the site at the new location and swing the A records around. The old location would stay up and would keep serving for a while, too, and then they'd have to somehow reconcile their logs and whatever later. Sometimes, they got clever and just proxied the stragglers from the old platform to the new. It was anything but simple.

More often than not, they'd just swing it around and then they'd write off the downtime. They took it as "one of those unavoidable things".

The big question for this half-baked idea is whether anyone would find it useful. Something like this would need a reason to exist or nobody would bother. I can think of a way how it might happen, but it's even more of a stretch than this idea is in the first place.

Imagine if the next big game console included support for this and did it as an open standard. Now, game players could connect to each other without worrying about chopping holes for specific ports or running horrible things like UPNP. Individual consoles would be directly addressable, even to the point of allowing multiple consoles per ordinary Internet connection. Any problems where having more than one running at the same time would fail due to NAT limitations would go out the window.

Obviously, the consumer router boxes of the world would have to add support for this, but I think if a bunch of gamers start yelling "shut up and take my money", the vendors in question will find a way to make a patch. Either that, or someone will step up with a new product. Given that some of these products already run DD-WRT right out of the box, how hard could it be?

So, to recap, here's the idea in a nutshell.

Take the existing IPv4 addressing scheme, and continue to use it to get packets between gateways. Then stuff 32 more bits into these packets to specify the "internal destination".

Hosts and gateways at either end have to change, but routers in between gateways can be blissfully ignorant.

...

Now for a few random on-topic thoughts while I'm writing about this...

Let's say you had clients which spoke this scheme, say, every smartphone running at least version X of your OS. Then you had a gateway which understood it. Then you had a whole bunch of old server machines which *didn't* understand it. It wouldn't matter, since the gateway would be able to do a sort of "directed NAT" based on the incoming traffic.

That is, you have a standard 2013 Linux box with IP 10.0.0.1, and new clients connect to X.X.X.X:10.0.0.1, your gateway rewrites that to just 10.0.0.1, and sends it along. The Linux box responds like nothing funny is going on, and the gateway rewrites it on the way out.

...

I guess clients would also need to have their internal addresses conveyed in here, too. This means another 32 bits crammed in there somehow. That would be the only way to convey the complete address: source gateway + source host, destination gateway + destination host.

...

It occurs to me that IPv4 itself has options for source routing. Granted, those are not the same thing as what I'm talking about here, and I suspect that most of the net won't support any of those options for reasons of security or other paranoia, but they are still part of the spec. Dig around in RFC 791 (September 1981!) and you'll find them.

It would be funny if a variant on that encoding was used to extend the addressing far beyond that which we can realistically use today.

So, let me be clear: the extra destination address in my idea is only to be interpreted relative to the inside interface on a gateway.

...

Finally, I know that sometime after I thought of this, I discovered that someone else had the same basic idea years before. Unfortunately, I haven't been able to find a single citation for it. I think it was called "ABCDE", but you try looking for "ABCDE" and "IP" without getting "a.b.c.d" and all sorts of other cruft in your results.

I'm beginning to wish I had saved a copy of that post or document from so many years ago.

I still have all of the source code from our attempts. I also managed to scrape together most of the problems. Some of them were easy. Most of them were not.

One problem was to print a giant "F". They give you a size and you have to render a huge letter F on the screen from asterisk characters. It has to be that many rows tall and wide. Obviously, this means drawing a vertical line and a horizontal line, but you also have to draw the cross bar of that letter as well. They tell you how to handle this: the middle bar should either be in the middle or just above it if that's not possible.

I think they further added that the size would always be greater than 4 and you didn't have to accept smaller numbers.

This one was simple enough. It was the sort of stuff I used to do back on my C-64 and VIC-20. Get the size, get the middle, then start a loop from 1 to size. On the first row and middle row, print "size" * chars. On every other row, print just one. That's it. We got this one just fine.

...

Another problem was a validator for alphanumeric data. They referred to them as "license plates" and gave some rules. They can be either three capital letters followed by 3 or 4 digits, or 4 capital letters followed by 3 digits. Anything else is invalid.

Looking at this now, in 2013, I go "oh, that's a regex". However, way back at the time, I knew of no such things. That sort of magic did not exist in my DOS and Turbo Pascal and QuickBASIC world. My solution reflects my ignorance. First, I do a simple length check, since I can throw out anything which isn't either 6 or 7 characters long.

Next, I start a breathtaking (in the bad way) series of nested conditionals. It looks like this:

if (ord(plate[1] >= 65) and (ord(plate[1]) <= 90) then
  if (ord(plate[2] >= 65) and (ord(plate[2]) <= 90) then

That's testing for something between A and Z (inclusive) using the ASCII values. It does this three times, then looks for matching values between 48 and 57 (0 - 9). If it gets what it wants, then it says "valid". Then there's another block where it looks for that fourth character possibly being A-Z, and then tests for three more digits. That's also "valid". Anything else gets "not valid".

I imagine this could be done with a shell script that does a read, an echo piped into egrep with the appropriate regex, and a exit-code test to echo the right thing.

However, because it was possible through the stupid brute-force method, we managed to submit it successfully.

...

It started getting harder. They moved into the "mathy" stuff.

This problem has them providing 8 numbers representing 4 (x, y) coordinates for two ditches. You have to tell where they intersect, and if so, where they intersect. They also mentioned that "they are guaranteed to not be vertical or parallel", and I'm sure that means something to the people who intuitively "get" this, but it meant nothing to me.

What's weird is that despite having no understanding of this problem either then or now, I have source code on my disk here which apparently solves the problem. I guess my teammate was sufficiently skilled in the dark arts of coordinate wrangling to tell me what to do. I'm lucky he was there with me, since by myself I would have been totally stuck on this one.

There's a lot of weird stuff in here. First I read in the four endpoints of ditch 1, then the four endpoints of ditch 2, this gives me d1end[1-4] and d2end[1-4]. Then it starts doing all kinds of crazy things. If d2end1 is greater than d2end3, then we swap them.

Then the tests start. If not (this) and (this) or (this) and (this), then "ni". In this case, that's not a shrubbery reference, but rather a call to a function which says "no intersection". This goes on for a while, with more and more tests.

After that, there are more swaps, and even more tests. They get bigger and bigger. Finally, down at the bottom of all of this, there's a call to "is" which just prints "intersection". The problem is that I see a bunch of debug scaffolding and nothing which would actually print the intersection. This implies we never completed it.

...

The last problem with any source code on my drive is all about massive numbers. They wanted you to raise 2 to a given power, and then return a given digit from it. If the power was 16 and the digit was 2, then the result would be 5, since 2^16 is 65536 and the second digit in that number is 5.

I'm pretty sure I knew about the basic limitations of the numeric types in my programming language of choice and knew it wouldn't be able to handle a bunch of crazy-big numbers. The code fragment left over on my old archive directory just asks for the input and starts doing something screwy with the "requested digit" value. I think maybe my math-clued partner had some idea involving reducing the workload based on which digit they requested, but it's been lost to the sands of time.

That's the last problem we attempted. There were several more in the contest, however, but we didn't even touch them. I did find the list of problems at some point so I can share what they are even though I have nothing in the way of code to show for it.

...

This one is like a one-dimensional Othello board. There are two players, each with their own color. On each turn, a player can place a single piece, and then it's the other player's turn. You can put a piece on any open spot. When your placement totally surrounds those of your opponent, they all become yours.

They provide some examples for this. If the board size is 5 and the board is "BW_WB", playing a B at 3 will turn the board into "BBBBB". The game is over and blue (B) is the winner.

If the board size is 6 and the board is "B_W_WB", playing a B at 4 makes it into "B_WBBB". The W at #3 wasn't surrounded and thus doesn't flip.

They give you the board size and the moves the players will make, starting with blue. You need to provide the final board as a string of Bs and Ws and tell who won and with how many pieces. Ties should also be reported.

Given that there's not even a hint of code for this thing, I'm going to guess we took one look at this, made some kind of "yeah right" meme face and flipped to the next page.

...

The final problem is completely nuts as far as I'm concerned. They introduce these "new integers" where they are a pair of positive integers like (7, 11).

Then they say the sum (a+b)+(c,d) is defined as (a+c,b+d), or:

(1,2) + (3,4) = (1+3, 2+4) = (4,6)

The product (a,b)*(c,d) is defined as (ac+2bd,ad+bc), or:

(1,2) * (3,4) = (1*3+2*2*4, 1*4+2*3) = (19,10)

The pound (a,b)#(c,d) is defined as (a,d), or:

(1,2) # (3,4) = (1,4)

It goes on like this. Next they say it's postfix/RPN (see, there's the head-hurting calculator business again...) where normal integers are separated by spaces and the operators follow operands. Your program has to take a line of ordinary integers and print the resulting "new integer".

-1 means +, -2 means *, -3 means # and 0 means = (evaluate the expression).

Then they provide a whole bunch of examples and sample runs I really don't want to key in.

If that 1D Othello-ish thing made me do a meme face, I can't imagine what my reaction to this must have looked like. I'm pretty sure I responded to this by saying to my teammate, "hey, want to see this thing I downloaded the other day?", and then we started messing with random junk on my machine.

That is, I think we ran out the clock doing things other than slaving away at some dumb programming contest problems.

Instead of talking about an actual product, allow me to use something mundane as an example. In this case, I'm talking about the lowly toaster. I am specifically talking about a toaster, which turns bread into toast, and not a toaster oven or anything else of the sort.

Think of what the earliest toasters must have looked like. I bet they were big cast iron affairs with long handles where you heated it up somehow, then slapped the bread in there and squished it down for a little bit. Then you'd have to pull it out of there before it burned.

Let's call that version 1: it involves brute force and actual fire. Eventually, they turned into actual electrical appliances. Call that version 2. The sort of thing you might have found in a suburban kitchen in the '50s or '60s should have represented a few decades of improvement. Maybe they had better timers, or handled more situations, or didn't burn down your house nearly as often. Call those version 3.

What's happened lately? Well, there are electronic timers instead of those spring-loaded clockwork things. They sometimes support different modes - "warm", "bagel", or just regular mode, for instance. Some have more than two slots and might let you choose when to use the extra slots so they don't waste power.

Maybe the best toaster you'd get today is a 3.5. They're the same basic idea as that version 2 electrical appliance, but they do a few more things. You could show it to someone who only had one of those really scary original electric models and they'd probably be able to figure it out.

Now let's say you're some kind of toaster visionary. You come up with a device which is so far "out there" that it would be like a 7 or 8 on this scale. It's so completely out there that I'm having trouble coming up with a description to make it work with this analogy. Maybe it's some kind of orbital toaster platform which beams microwaves through your roof and focuses them on your plate. All you have to do is put the bread on a plate and hold it over your head like you've just been given a sword in an adventure game. The orbital toaster thingy does the rest.

Nobody is going to understand this. It's just too far out there. Realize that most of the world, assuming they even have a toaster, is sitting at a 3.x, or maybe even a 2.x if they inherited some scary 1950s thing from their parents and never bothered to replace it. They might be able to handle a 5 as a stretch (whatever that would be), but what they really want to see right now is a 4. To most people, a 7 or 8 isn't even the same kind of product, and they can't see how it would ever apply to them.

The problem is that you're a toaster visionary and a 7 or 8 is what you're going to create. You've already seen your way past versions 4, 5, and 6 in your head. It's just like when you don't write down some steps in a long math problem because you "just know it". It's second nature when you start creating things. You don't stop off at the intermediate points because you already know what comes after that.

However, being technically correct and being successful in the marketplace are two different things, even when no money is involved. You just have to inch along and dole out the features piece by piece, year by year, maybe even with the "tick, tock" strategy. Do you think Apple really only came up with the finer points of the iPhone 5 in the year or so before its release? I doubt it. I bet they had all of this figured out way back in 2007 when they released the first one, and have been slowly moving the "window" through successive updates.

Besides, this way, they get to sell way more devices and get all kinds of money from it. It's a double-shot of awesome in that case, since people actually "get" these marginally-improved devices, and then they go and buy a replacement every year or two, whether they need it or not.

The biggest hang-up I have with this is that it seems to require deliberately "de-rating" your own output in order to be successful in the market. That just seems cheap, and borderline unethical to me. I mean, why would you possibly do anything but your best work at any given time, barring some kind of illness, injury, or some other external limiter on your abilities?

Is this really what it takes to make a successful product?

If you use "zite", also known as "woriobot" (whatever those are) and don't get images for my posts, it's because their robot is badly broken. Check this out.

"GET /dialup1.jpg HTTP/1.1" 404
"GET /dialup2.jpg HTTP/1.1" 404
"GET /tc.jpg HTTP/1.1" 404

Those three images are part of my terminal server post from Wednesday. If retrieved as a post (that is, directly to /w/2013/04/17/slow/), it refers to those images using relative paths. That is, they are just "dialup1.jpg", with no dots, slashes, colons, hostnames, ports, protocols, or anything of the sort.

That means it lives at the same place as the base URL. Since I don't twiddle that setting in my pages, that means it's the same path that they just fetched plus the "dialup1.jpg". Easy. You'd think this fundamental tenet of the web would be well-understood by now, but apparently it is not.

Now, let's say they're actually crawling my Atom feed. That feed purposely spells everything out in long-form: protocol, hostname, path, filename. This has been the case since September when I declared that my "protocol-relative URL experiment" was over.

They aren't alone in their brokenness, though. There's another one from "Sosospider" which is has its own flavor of insanity:

"GET /w/2013/04/02/maps/img_0725.png HTTP/1.1" 404
"GET /w/2013/04/02/maps/img_0730.png HTTP/1.1" 404

These are from my first post this month about bad Apple maps. The problem is that these files are *uppercase*. They are the same filenames which came straight out of my old iPhone's screenshot facility. The file isn't called img_0725.png. It's IMG_0725.PNG.

I'm serving up HTML with the proper filenames. Nearly everyone manages to get this right. These guys, however, squash the case and so miss out. Who thought that was a good idea? Wouldn't it take more code to purposely squash case, and properly at that? Getting uppercase and lowercase right is just like date handling. Both are hard problems.

This is all on top of the well-known Java crawler stupidity where someone has decided that parsing the URLs which are targeted by SCRIPT tags is a good idea, even though those are not even HTML.

Welcome to the web, where any idiot can program for it, and probably does.

Tuesday, April 7, 1992 probably started like any other morning. I probably did whatever I normally did to get ready for the day, and that included sitting down and having breakfast. Then, partway through my breakfast, the entire house started shaking. The windows were rattling in their sills. The roof was making "bumping" sounds. Nobody knew what to make of it.

We turned on the radio and heard the same thing from a bunch of DJs. They had all heard or felt something, too. One of them said it "sounded like someone was running around on the roof at the station". Nobody knew what had happened, but it was clear it had basically affected the entire city... of Houston. Yes, it was a big place, and lots of people felt whatever it was.

Later that day, everyone started hearing what had happened. There had been a leak of liquefied petroleum from a pipeline near a salt dome storage facility outside Brenham. All I knew about Brenham back then was happy cows and Blue Bell ice cream, but apparently they had a whole bunch of petrochemical storage stuff going on, too.

This leak had been ignited somehow, and the resulting explosion sent out a groundwave that made my house rock and roll in Houston... 70 miles away. Apparently people could hear it 140 miles away.

News reports put this man-made earthquake at between a 3.5 and a 4 on the Richter scale, which is what they were using at the time. I guess, in that sense, it was my first earthquake.

A "20 years later" followup from last year tells the story far better than I could.

I've talked to other people who lived in the state back then, and possibly were even rattled by it, and have discovered they don't remember this event. I guess some things are better left forgotten.

For those interested in surveying the area as it exists now, don't try to use the crossroads given in the news stories. The road names seem to have changed. Instead, just look at this address: "9800 Oil Field Road Brenham, TX 77833". It still shows up as "Seminole Pipeline".

If you have any doubt, just switch to a map which shows property lines. That long, skinny parcel cutting from west to east through that whole area sure says "pipeline" to me.

"Duck if taller than 9 feet!" ... and goose if shorter?

I've made snarky references to biking on a freeway in old posts. This guy went and did it.

Yep, that's a bike on 101 through Mountain View during rush hour. I think I actually saw this happen more than once, too. Nutcase.

So let's review: I'm on some kind of Android device, in fact on a device given to me by Google, running nothing but Google software, talking to a Google web site, and it says the browser isn't supported? Plus, it has lots of awesome C escape sequence damage.

Ah, you're either a Fraggle... or you're a Doozer.

Finally, there's this, spotted in the Target frozen foods area:

The word "meat" in a food context is always amusing to me. Normally you see things labeled as a specific type: chicken, beef, pork, whatever. When they punt and label something as "meat", it always makes me wonder. Did they not want to tell people what it really is? Do they even know?

This isn't quite the same thing, but there is a certain "caveman" element to calling stuff just "meat snacks". It makes me wonder exactly what they had there: it's meat, it's a snack, and it's frozen? I did not know those three things converged.

Meat + snack? Okay, it's jerky or similar.

Meat + frozen? Pepperoni pizza, maybe? Premade meatballs?

Frozen + snack? Popsicle? Ice cream treat?

Now combine them. Meat + frozen + snack? What is that going to be?

[17/Apr/2013:22:33:40 -0700] "GET /w/atom.xml HTTP/1.1"
[17/Apr/2013:22:33:40 -0700] "GET /w/atom.xml HTTP/1.1"
[17/Apr/2013:22:33:40 -0700] "GET /w/atom.xml HTTP/1.1"
[17/Apr/2013:22:33:40 -0700] "GET /w/atom.xml HTTP/1.1"
[17/Apr/2013:22:33:40 -0700] "GET /w/atom.xml HTTP/1.1"
[17/Apr/2013:22:33:40 -0700] "GET /w/atom.xml HTTP/1.1"
[17/Apr/2013:22:33:40 -0700] "GET /w/atom.xml HTTP/1.1"
[17/Apr/2013:22:33:43 -0700] "GET /w/2013/04/17/slow/dialup1.jpg HTTP/1.1"
[17/Apr/2013:22:33:42 -0700] "GET /w/atom.xml HTTP/1.1"
[17/Apr/2013:22:33:42 -0700] "GET /w/atom.xml HTTP/1.1"
[17/Apr/2013:22:33:42 -0700] "GET /w/atom.xml HTTP/1.1"
[17/Apr/2013:22:33:42 -0700] "GET /w/atom.xml HTTP/1.1"
[17/Apr/2013:22:33:43 -0700] "GET /w/2013/04/17/slow/dialup1.jpg HTTP/1.1"
[17/Apr/2013:22:33:42 -0700] "GET /w/atom.xml HTTP/1.1"
[17/Apr/2013:22:33:43 -0700] "GET /w/2013/04/17/slow/dialup1.jpg HTTP/1.1"
[17/Apr/2013:22:33:42 -0700] "GET /w/atom.xml HTTP/1.1"
[17/Apr/2013:22:33:42 -0700] "GET /w/atom.xml HTTP/1.1"
[17/Apr/2013:22:33:43 -0700] "GET /w/2013/04/17/slow/dialup1.jpg HTTP/1.1"
[17/Apr/2013:22:33:42 -0700] "GET /w/atom.xml HTTP/1.1"
[17/Apr/2013:22:33:42 -0700] "GET /w/atom.xml HTTP/1.1"
[17/Apr/2013:22:33:43 -0700] "GET /w/2013/04/17/slow/dialup1.jpg HTTP/1.1"
[17/Apr/2013:22:33:42 -0700] "GET /w/atom.xml HTTP/1.1"
[17/Apr/2013:22:33:43 -0700] "GET /w/2013/04/17/slow/dialup1.jpg HTTP/1.1"
[17/Apr/2013:22:33:43 -0700] "GET /w/2013/04/17/slow/dialup1.jpg HTTP/1.1"
[17/Apr/2013:22:33:44 -0700] "GET /w/atom.xml HTTP/1.1"
[17/Apr/2013:22:33:47 -0700] "GET /w/2013/04/17/slow/dialup1.jpg HTTP/1.1"
[17/Apr/2013:22:33:47 -0700] "GET /w/2013/04/17/slow/dialup1.jpg HTTP/1.1"
[17/Apr/2013:22:33:47 -0700] "GET /w/2013/04/17/slow/dialup1.jpg HTTP/1.1"
[17/Apr/2013:22:33:47 -0700] "GET /w/2013/04/17/slow/dialup1.jpg HTTP/1.1"
[17/Apr/2013:22:33:47 -0700] "GET /w/2013/04/17/slow/dialup1.jpg HTTP/1.1"
[17/Apr/2013:22:33:47 -0700] "GET /w/2013/04/17/slow/dialup1.jpg HTTP/1.1"
[17/Apr/2013:22:33:47 -0700] "GET /w/2013/04/17/slow/dialup1.jpg HTTP/1.1"
[17/Apr/2013:22:33:48 -0700] "GET /w/2013/04/17/slow/dialup1.jpg HTTP/1.1"
[17/Apr/2013:22:33:48 -0700] "GET /w/2013/04/17/slow/dialup1.jpg HTTP/1.1"
[17/Apr/2013:22:33:48 -0700] "GET /w/2013/04/17/slow/dialup1.jpg HTTP/1.1"
[17/Apr/2013:22:33:48 -0700] "GET /w/2013/04/17/slow/dialup1.jpg HTTP/1.1"

It's not an isolated event, either. They seem to love pulling atom.xml a dozen times in a 5 second period several times a day. Hey Feedspot, get it together, already.

I made a reference to the time I had to install a terminal server (dialup modem pool) at my school district sysadmin gig in my "crack pipe" followup post on Sunday. This is the rest of that story.

This story starts a number of years before a Lucent tech said I was "smoking crack". It started with a standalone box running BSD/OS which had a Digiboard attached to give it 8 external serial ports. Each of those ports had a 28.8K Sportster attached -- these were the old white ones which looked like glorified office intercoms.

We had the "phone tech" guy add a bunch of analog ports to the PBX and run them over the wall to where that particular box lived. They all got their own extension and then it had a "leadin" number which would roll over to the first available extension. This all worked fairly well, but it was a serious mess. Just look at this situation:

There's my original Unix box at that job: a "mighty" Pentium 90. Note the explosion of analog ports on that wall along with a bunch of Ethernet drops. Then there's all of that "flat satin" telco cord mess along the right side. Finally, there's a lonely "ZOOM" modem on its side next to the box. That was our Internet connection at the time: 28.8K PPP to the outside world. Really.

The Digiboard and modems are just out of frame in this picture. A couple of years later, I went in there and cleaned up that mess in order to fit a few more boxes in that same spot. Now it looked like this:

I don't have a better shot of this, unfortunately, so I will just describe the situation. There are now three Unix boxes in this picture: the original, its replacement, and another one we added for web serving and proxying duties.

A sharp-eyed viewer might notice the modems have some kind of black marks on them now. This is because I broke out a permanent marker and labeled every single modem, power brick, and RS-232 cable with the tty name. This way, you would know exactly which line you were messing with if you had to disconnect something for some reason. I may have even labeled those flat satin phone cords, for that matter. Those cords were even bundled up with rubber bands to keep them from going crazy.

So, why is the old box there if it was replaced? That's easy. BSD/OS didn't get along with the Digiboard on the new hardware. Any time you started generating a lot of serial traffic, it would freeze the entire machine. I couldn't figure it out, and their tech support was of no use either, so I wound up moving the Digiboard back to the old box, and just left it up for the duration.

This meant I had one box the users would actually dial into, and another one which had their shell accounts and e-mail and all of this. I had to train them to run a special command to telnet (yes, really) into the new machine when they logged in in terminal mode. My users who did PPP or whatever weren't affected by this, fortunately.

The dialups were becoming more and more popular. We didn't want to deal with punching even more holes in the wall and having yet another stack of modems. While both the Unix box and the Digiboard would have supported another 8 easily, the resulting pile of parts would have been a disaster. There was already a decent amount of heat coming off all of those things, and cramming more in there wasn't going to happen.

This ultimately got me looking at terminal servers. In theory, we'd just buy a nice box, put it in the data room where it belonged (instead of out in someone's office), and give it some T1s. Then it would handle users for me and I could finally retire my aging original dialup box. The plan was simple enough: get a card for our PBX which would give us a PRI circuit, then come out of that and drop into a Bay Networks terminal server.

I went out and talked to vendors and had them do their dog and pony shows for me. This got really messy at times since we had strange working relationships with certain companies. USWest was the local telco, and they used to sell us both Lucent and Bay equipment. One day I wound up on a conference call with my boss and two Lucent people, and this one guy would not shut up about how amazing their Portmaster terminal server was. They had just purchased Livingston and now it was the new hotness. They didn't want to talk about Bay terminal servers any more.

I finally had to make him stop talking and just told him "From Lucent, I want the stuff to get PRIs out of our switch. From Bay, I want the terminal server. That's it."

I had made the decision about the terminal server based on its specifications and our needs. For one thing, we had an installed base of users who had modems which spoke the USR-only X2 56K standard. We therefore wanted something which would also speak that on our side. That either meant a 3com device (since they had eaten USR some years before), or the Bay device which also supported it. The Portmaster was right out.

After all of this, I finally had a proposal put together, and then we found out the FCC killed the funding for it. This was one of those "E-Rate" things where they would pay some percentage of the overall price based on how many of our kids were on the free or reduced price lunch program. Apparently they had said terminal servers were "no longer being funded", and so that killed the project.

My box in the corner with the modems shuffled on. My users continued connecting at 28.8K or thereabouts. They were disappointed.

Months passed. I found out there was some kind of "appeals process", and was told to go through all of this again. In the time since I had last worked on this project, new products had come along. The field had changed yet again and so I had to reconsider everything. More vendors came and went, and more of them tried to sell me Portmasters, not realizing that I needed X2 and V.90 support.

Finally, a couple of them clued in and started talking about the 3com Total Control stuff. They had just done something in that product line which made it far simpler than before. Now, a single card would do both the T1 interfacing and the modem stuff. Previously, you had to buy line cards, and modem cards, and all of this.

The new situation was simple enough: one chassis, two line cards, a management card, and the actual terminal server card. We went out for bids, got the responses, chose one (probably the lowest bidder), and placed our order. Yes, we had finally gotten to the point of exchanging money for a product.

A week or two later, this gigantic box arrived. I opened it up at my desk and was rewarded with an insanely heavy device intended to be mounted in a standard 19" rack. I, of course, plugged it in at my desk to make sure it was sane before attempting to rack the monster. It seemed healthy enough, so into the rack it went, both to get it out of the way and to preserve my hearing - those fans are loud!

A couple of weeks went by. We were waiting on parts for the switch. This was the other side of this project. I had the "terminal server" side, and the local phone switch monkey type had all of the PBX side. His job was to deliver me 46 extensions in PRI form into the jacks on the back on my device. My job was to take them and provide awesome dialup service with them. It was a simple division of duties.

The day finally arrived for us to start connecting things. It was a Wednesday. Along with a tech from 3com, we found that one of our cards had failed. It was either the network management card or the terminal server -- not either of the line cards. Those were just fine, and I could log into them through the serial port on the back. The telco monkey overheard this and took it upon himself to send the Lucent techs home because "we have a bad card". The bad card wouldn't have affected the T1 bringup, but he used any opportunity to screw with me.

3com shipped us a new card overnight, and the next day (Thursday) their tech returned to do the install. The Lucent guys showed up and we tried to get it going, but it wouldn't happen. That's when the resident telco monkey told me "our switch might not be able to do PRI". Well, gee, you fool, what have we been talking about for the past year? Doing PRI from your switch into my terminal server to do dialups. It seems like a halfway competent person would have checked for PRI support before accepting that side of the project, no?

I only thought that. I didn't say it. Sometimes I wish I had, though.

Anyway, he said a Lucent guy would come out the next day to get things going. I figured that would be the day we turn this thing up. Ha. Yeah, sure, right.

Friday morning, I showed up, and got word from one of the "network engineers" that the telco monkey guy left a voicemail on his extension. He said he wasn't going to be in today. He also added that he didn't leave a message on my voicemail because "he didn't know the extension". Yes, really. The telco guy didn't know the extension to someone in the same department and apparently couldn't figure out how to use the directory feature, either. The one he set up.

At any rate, the Lucent guy was still supposed to show up that morning. 9 AM came and went, and no Lucent guy was seen. Around 9:30, I walked around to the far ends of the building to ask people if they had seen a lost-looking tech in the area. They said no.

About then, I found out that the telco monkey guy was taking the ENTIRE next week off. The "network engineer" guy who had received the voicemail actually had scheduled a meeting with him and a 3com guy for some other reason in the afternoon, and now that had to be rescheduled.

Of course, the boss was on vacation during all of this, so there was no way to tell him what was going on. Both the boss and the telco guy were slated to return on the same day: Monday, two weeks out.

At this point, the 3com installer guy had to leave. He obviously couldn't come out again, and couldn't wait around due to our stupidity. I signed off on his little paper which said he did what he was there to do, and sent him on his way with my thanks. It was up to me to handle the rest.

Several more weeks passed. Then, one day, I got to spend the entire day on the phone with 3com tech support trying to get things working with whatever the local telco people were setting up on their switch. I'd get it to start syncing, and then it would change again. I had told them what I wanted, and they never delivered it. This is the "smoking crack" part of the story I told back on Sunday, for what it's worth.

Eight days later, somehow, they finally managed to get it going on their end. They punted back to channelized T1 instead of the PRI I wanted, but I'd take that over nothing at all. They also only gave me one circuit, but again, that was better than nothing. I got all of my stuff configured and started making test calls across the PBX.

I added a test account by hand and started poking at it. I was getting "50666" connections dialing the four digits into my brand new terminal server. Typical latency across the link in PPP mode was 90-100 ms as measured with ping. Life was pretty good. I set to work on RADIUS and things of that nature and ultimately got both PPP and ordinary interactive "shell" logins working too for my normal set of users straight from my usual passwd/shadow files.

Finally, I sent out a mail to my ever-so-patient users, letting them know the new system was in and that I would accept some people who were willing to try "beta testing" my new setup. They'd have to change some settings temporarily and might have to deal with oddities while I figured everything out, but otherwise it was up to them.

Happily, a bunch of them signed up and I switched them to the new temporary lead-in number for the 24 lines which worked at that point. They loved it. It was fast, and solid, and never had busy signals.

A couple of days into this, they got the second T1 running and delivered it to my stuff, so I set that up too and now had 48 lines available. The users were in heaven.

It ran in testing mode for a couple of days, and then I had the telco guy switch the lead-in number around. The old number would now go straight to the terminal server, and the old analog extensions were disconnected. I finally got to undo all of that junk and removed it from the corner. The old box was shut down at last.

It took nearly a year to go from "we need this" to "it's done". This is the final result, mounted in the rack and running after far too long:

...

I have one more story about this telco guy who worked there. It's too short to warrant a separate post so I'm including it here.

There's something called "dialaround" which lets you pick your own long distance carrier regardless of who's been set up as the "1+" default provider (also known as the PIC). Back in the '90s, one way to do this was with five digit codes which started with "10".

"10288" would give you AT&T because "2 8 8" is ATT. "10222" was for MCI -- no idea why. "10333" was Sprint -- same thing. There were all sorts of codes. Due to the way these companies tend to split up and merge and split again, a bunch of them had multiple codes. Some of them worked, others didn't, and some were entirely variable.

Obviously, with just three digits (and some restrictions on which three), there wasn't an unlimited amount of space for new participants. So, at some point, the powers that be decided to expand the space. All of the existing codes would now have another "10" inserted. This meant "10288" would now be "10-10-288". "10222" was now "10-10-222". You get the idea.

This was announced well in advance of any changes. Then there was a "phase-in" period when both the five and the seven digit forms would work. Then, at some later point, the old codes were shut off, and that was that.

I had heard about this. Apparently our phone guy had not. Why do I know this? Easy.

The first day after the "phase-in" period ended, nobody could make long distance calls at the school district. We had this arrangement where dialing long distance would give you a tone, and you had to enter a special code for billing and tracking purposes. Everyone had their own code. Well, on this day, we weren't getting that tone. We were just getting an error.

As near as I can tell, he totally missed what must have been an avalanche of notices and just kept on going the way things always had been. Then, the day it actually broke, he had a huge fire to put out. There were important faxes which needed to be sent to the state, and all of those government numbers were in another area code! Then there was ordinary vendor business with companies who were out of state and didn't have toll-free lines. You get the idea.

I made an offhand remark about "probably needing to just add '10' somewhere", but he didn't comment on it. I don't know if that's truly what happened, but the timing was just too perfect for it to be something else.

Unlike programming and sysadmin environments, there was no way for the rest of us to log into the switch and see what kind of work he was doing. We just had to take it on faith that he was doing the right things at the right time. You can see how well that worked.

Here's my latest example: "full stack". By itself, what does it mean, exactly? I keep running into posts and stories and people using this term to mean approximately "I can write stuff to talk to the database in the flavor of the month language and I can also write client-side pages and scripting, too."

I guess that's how some people want to say that they do backend and frontend work at the same time. That's all well and good, but calling that the "full stack" seems like an abuse of a term which suggests something far more. When I think of that term, I imagine two far ends of the spectrum. One end has a pile of hardware, a box of cable, a bag of RJ45s and 8P8C connectors, crimpers, punchdown tools, toners, probes, and more. It's the grungy bare-metal stuff which makes the rest happen.

At the other end, I imagine someone who can sit down with a client or customer and figure out what's going on. They can handle support requests and empathize with their users. It's all of those "softer side" traits which aren't commonly found in the entire set of engineers.

Then, in between, I imagine the sysadmin and programming skills and everything else in those realms.

But hey, that's just me. Someone else will probably have a totally different definition of the term. That seems to make it of dubious utility as a descriptive term in the absence of any context.

Want to see how difficult this can be? First, figure out the major components of the "stack" as far as you're concerned. Then start breaking it down into its constituent parts. Don't forget about all of the "meta" stuff which goes with certain tasks. Do you think someone should be able to deal with that stuff as well?

Here's just one example. If you work in certain kinds of business environments, you have to put out a request for bids when purchasing certain devices. I had to do this as part of ordering certain large things at a public school district, for instance. Anything over $10K had to be handled this way.

If your idea of the "stack" includes hardware, does it mean purchasing in addition to assembly? If so, does that include dealing with vendors and things of that nature? Just how "full" is that stack, anyway?

I don't know if this one can reasonably be answered.

One of them didn't let me go through the usual "self-service" process. Normally, you'd just go out there and drop in your key, and it would let you into the system. This time, it didn't work.

I asked around and found out that other people who had started around the same time as me also were having problems getting into the system. Around this time, I noticed my employee ID was up around 33000. In other words, it was just a bit above 32767. The same applied to anyone who had started in the last month or two.

I never heard an official explanation, but I wouldn't be surprised if they had used a signed 16 bit value for some kind of ID field. Once it hit 32K, that was the end of the usable positive number space!

My account did eventually come online, but something peculiar happened. Instead of having ID number 33000-something to match my employee ID, it was actually 3300-something. I guess they just sliced off the last digit and found that the original #3300 person was gone, so they let me have that person's slot. I guess it was easier than changing their schema.

I can only wonder how long they had to do this hack job. Thinking about it now, this should have been a warning flag about the design practices of certain systems, but I wasn't looking for that sort of thing at the time.

"GET /w/2013/03/31/snark/filesystem.png"><img HTTP/1.1"

Got that? It's actually picking up a quotation mark, a greater-than, and then a less-than, and the beginning of an "img src"!

I finally figured out what was going on this morning. They've been fetching my protofeed file and have parsing it as if it was HTML! Yes, my half-baked protobuf-based feed file from last month has been linked a few times, and it started being indexed. Then, for some reason, they decided the blobs of text within that binary protobuf were indexable, and went to it. The result is that mess above.

I will note that I have been serving it as "text/plain" for lack of a better MIME type. It's definitely not going out as "text/html", in other words.

For now, I've "solved" it by blocking this file in robots.txt. Let this be a warning to anyone who links to binary data from their web pages. If you have something resembling HTML in that binary blob, they might start following the links, and this is probably not what you want.

There were many points raised in that HN comment thread, and I figured I'd respond to some of them here.

...

Various folks commented on the "being mistaken for HR" thing. I guess I need to give an example of how I make a business call. It seems to be something of a lost art.

First, I pick up the phone. Then I dial it. It rings. Someone picks up, and says "hello" or something to that effect depending on their locale. It's my turn to speak, so I do.

Hi, this is Rachel calling from $company. May I speak to $name, please?

In doing that, I have totally defused any possibility of it being the all-too-common annoyance call from a telemarketer. They don't have to ask who I am or what I want. I've laid my cards on the table and they are in control.

At this point, I'm usually talking to "$name", but sometimes, whoever answered the phone has to go fetch that person. Now and then, they would ask what I'm calling about, but that was mostly when I called businesses. It didn't happen when I did interview phone calls, since those were all (hopefully) direct to the candidate's personal phone.

Compare that to the (legitimate) business calls I seem to get now:

My phone rings. I look at it. It's some random number which has no mapping in my phone, or worse, "Blocked". I pick it up and say "... hello?", not knowing what to expect.

The person on the other end immediately asks if they can speak to Rachel. Well... I'm not going to say if they can or not until I know who they are and why they are calling this number. That forces things to go into the state where they say "I'm X calling from Y (about Z)".

Now you can see why I open with that. It cuts out that whole pointless and annoying exchange.

Does this sound familiar? If so, you might have read it in the book, or as a post right here in August 2011.

Does this set me up for having my engineering abilities doubted by the candidate? That seems completely wrong. Why is it so hard to believe that I could be technically knowledgeable, qualified to assess a candidate, and yet also know how to make a phone call "like a boss"? Don't people learn phone calling skills from their parents any more?

Actually, don't answer that. I bet a lot of people now grow up using their "phones" without realizing they can do voice-based comms with them. Grumble grumble.

...

Apparently I'm a "smart-ass know-it-all whiner". Well, you might think that if you only ever read one post which was talking about horrible things which had happened. Go read some of my more technical posts and you might change your tone.

As for being a smart-ass? I reserve the right to be that way when facing off against a bunch of ramrods. So there.

...

Quite a few comments said something like "it happens to me, too!". My immediate reaction to that is "and it sucks, doesn't it?". Still, these are two different assertions. I'm saying that something seems to happen to someone in set A. I'm not saying anything about whether it also happens to people outside set A. Those people have spoken up, and okay, so it does. Now we know that the problem space is a bit larger.

Let's try looking at this another way with a thought experiment.

Suppose the police chief in a city hates people from outer Elbonia. It's completely irrational, but that's just the way this person rolls.

It just so happens that everyone from outer Elbonia drives yellow cars. It's a long, drawn-out back story involving the local culture which would take forever to explain. Don't worry about why it happens. It just does.

Other people also drive yellow cars for whatever reason. They aren't from Elbonia. They might not even know anyone from there. Maybe it's a perfectly random distribution of non-Elbonians who happen to drive yellow cars.

The police chief decides to start "profiling" anyone who drives a yellow car to further his own agenda, and tells the patrol units to watch them very closely. If they drive 36 in a 35, pull them over for speeding. If they scratch their head while driving, pull them over because they might have a cell phone in that hand, and that's illegal. You get the idea.

So this starts happening, and pretty soon, all of the Elbonians are being pulled over fairly regularly, and are being picked on by the local police force. It's a pretty crappy situation.

Meanwhile, the non-Elbonians in yellow cars are also being pulled over. It's pretty crappy for them, too.

One day, some Elbonians speak up and talk about what's been happening, and shares some stories involving their people. It's clear something screwy is going on, and they want it to stop.

That's when the randomly-distributed yellow car drivers chime in and say "it's not anti-Elbonian!" Well guess what? You're technically correct. It's actually technically anti-yellow-car. However, the driving force behind that is actually something sinister: the police chief hates people from that fictional country.

Is this starting to sink in yet?

Human traits overlap. The set of traits which frequently earn bad treatment at the hands of others in this industry seems to largely overlap the set of women. However, it isn't a 1:1 mapping, so you will have women who don't have those traits, and men who do, never mind the rest of humanity.

This doesn't mean the situation should be allowed to continue.

I can draw this out as a series of Venn diagrams if people really don't get this. Come on, already.

...

As for the actual "crack pipes" line? It was used on me once by a Lucent tech. Way way back when, I was trying to get a terminal server turned up at one of my gigs. I had ordered it with two T1 cards so we'd have 46 dialup lines. Yes, 46, not 48. I wanted to do PRI instead of channelized T1 with robbed-bit signalling. This way, my users would have better X2 and V.90 performance. I didn't need the two extra lines, especially considering that my existing modem pool was an actual stack of 28.8 Sportsters. (Yes!)

Well, the people who worked on that phone switch at this place where I worked weren't exactly clueful. They were cargo cultists in every sense of the word. If it wasn't something they had done a million times before, they didn't know how to do it and really couldn't figure it out. That's why the Lucent guy was there. He was supposed to help our guys figure out how to add a pair of circuits out of that switch which would run the six feet into my terminal server.

I found that things just weren't stable. I had provided the requested parameters for the line, but they didn't deliver them consistently. I'd manage to get it to sync by flipping things around into other configurations, but then they'd change it again. They wouldn't just set it up and leave it alone, and I was following them around. It was madness.

Well, at some point, the two of them (our phone guy, and the Lucent guy) came back to where I was sitting, and I told them what was going on, and asked them to just turn it up the way I had requested. The Lucent guy looked at me as if he didn't believe what I had been saying (about the line coding changing) and said something like "sounds like someone's smoking crack", and they both chuckled, turned, and walked away.

It took well over a month to get that stupid thing running thanks to them.

I could write a whole post about this saga. Maybe I will some day. The point is, people have used the 'smoking crack' thing on me, too.

...

Someone said I should "dress nicely". Okay, that is definitely a mine field, particularly deep within the tech nerd-dom which is the valley. If you show up wearing a t-shirt, jeans and sneakers, you will look like nearly everyone else. Parity is achieved. Yes, that t-shirt and those jeans are curvier than others in certain places and the sneakers might have some pastel highlights, but at least you don't stand out too much.

The first time you go up a notch, and *gasp* wear a skirt or *shock and awe* go all the way to a dress, people will notice, and they will comment on it. You also run the risk of being seen as not an engineer by people who don't already know you... and maybe by a few who do. This has a way of limiting your wardrobe choices, no matter how beautiful the day may be.

I suspect that trying to command technical respect from those guys by varying my wardrobe in the "more formal" direction would have only ended in failure.

This didn't stop me from going beyond the usual engineer uniform many days, but that was because I wasn't going to give in on that part of my life. Who knows. Maybe it contributed to the overall lack of respect I got sometimes.

Meanwhile, if a guy showed up in something a little more formal than usual, like, say, wearing a shirt with a collar, it was universally assumed that he had an interview for another job. Even if he didn't, you'd joke that he had just come from one.

Such things are completely disconnected from ability in that world.

...

I didn't see many suggestions to "be more like the guys", but to any guy who thinks that way, let me give you a suggestion in kind. Try being like a vastly different sort of guy for a month and see how well that works out for you.

If you're usually a rough, tough, cowboy type, try being a soft or effeminate type for a month and see how you feel by the end of it. Likewise, if you are naturally soft, go rough and tough for the same time. Change your mannerisms, clothes, and how much space you take up in the room. Defer more... or less. You get the idea.

(My apologies for not providing more example types. I am not well-versed on the particular variants of your tribe.)

Whatever your thing is, flip the sign on it for a month -- no cheating! You have to be this way any time you're involved in work. Even if you're at home writing e-mails, you have to come across this way in those mails.

I suspect you will find that "inauthentic" behavior is rather draining.

If you're one to laugh this off, but at the same time still recommend that someone acts like something they're not, then you're just a hypocrite. Try taking some of your own medicine and see how it tastes.

...

"A professional does not lose her cool or insult others".

I have two things to say to that. First, a professional does not ignore teammates.

Second, calling someone unprofessional is basically the last-ditch option when you can't come up with anything else to say. It's a pretty good indication that the speaker is grasping at straws.

...

Someone made an aside about food at Google being free. Sorry, but it definitely is not free.

What's funny is that the IRS has picked up on this in recent days. All I can say is: you brought this upon yourselves, you evildoers.

...

There was a link to a clip which shows the exact thing happening.

Go watch it. It's short.

If what happened to me was an isolated event, then it would be rather odd for some Hollywood writer to notice and put it into a script, no?

...

Someone related a tale of not being taken seriously because they were young. I've seen this happen to plenty of knowledgeable youngsters, and it also stinks. Just because someone is 12 doesn't mean they might not know what they're doing. They might have a totally fresh angle on something... or they might be a crackpot... just like anyone else. You have to listen to them to find out the difference.

The thing is, those kids grow up, and thus "earn" the respect they should have had all along. They leave the youth behind.

That doesn't work when the problem is one of gender.

...

Okay, that's enough for now.

In recent days, we created a little test program to chew on QUERY_STRING data to deal with the "?foo=bar&blah=1234" stuff which comes from submitting a form as a GET, or just building a URL that way by hand. Then it was turned into a library and picked up a bunch of tests.

After that, it learned how to deal with percent encoding, which is how you can have things like ampersands and equals signs in your data without messing things up.

Now it's time to finally use this library within a bigger program to get something minimally useful to happen. In today's set of recordings, I write just enough of a wrapper to use the CGI library and grab a specific variable name. Then, depending on what it's set to, I respond with the system time or load average as appropriate. Finally, I show how to write a small HTML page with a form which will let you pick "time" or "load average" and will send it to the new program.

If you ever wondered how to get some dynamic content into a web page, this is the ground floor. Start here and you can work your way up. Soon, there will be POST methods, AJAX calls, and all sorts of other shiny new things which power the web of today.

All are welcome, so feel feel to stop by any time.

I watched as some random people from a part of the company I had never even heard of before got up on the stage and started giving a presentation. It was some big shiny polished affair with slides and a pair of people who would switch off just like a couple of newscasters. There was one part of this which didn't make any sense to me.

They used the word "ideate". I don't just mean once, and just that form, but rather, many times, and in various forms. I'm talking about probably a dozen different cases of throwing that in there while they were blathering on about whatever it was.

The first mention woke up my corporate BS detector and the second mention activated it. However, by the tenth or eleventh instance, I was just beside myself. What kind of hell have we descended into where this kind of corporate stupid-speak is being taken seriously? Where are the engineering roots which supposedly founded this company?

I had to get up and leave. It was just horrible. Back at my desk, I posted something to the internal space I had been using for muck-raking and generally snarking about dumb things. It was effectively an open letter to everyone in general, asking them to kindly knock it off -- that is, drop the corporate nonsense talk!

There are plenty of ways to talk about things. However, if you choose a certain subset, you can basically say "I spend all day in meetings and do nothing else here". These people were doing exactly that, and I couldn't possibly take them seriously.

I mean, come on, are you honestly going to say that a bunch of people "had an ideation" about something and that your new project "enables them to ideate"? They did... over and over.

Then again, someone once told me that we should have a "groupthink" about something, so there's that. It's like we're using the same words but are speaking two completely different languages.

This is a small collection of stories from when they tried to roll it out, plus some other insanity these guys foisted upon their users.

...

One day I showed up and there was a brand new 21 inch monitor on the desk of one of the "network engineers". This was back in the time of CRTs, so it was absolutely massive. As he explained it to me, "I need a bigger monitor because the remote screen's resolution has to match". Apparently scaling wasn't an option for them. I don't know if this was a limitation of SMS or just a limitation of the humans who were running it.

Something else they told me around then was how most of the systems were running at 800x600 because the school district management software called "Phoenix" would break at anything higher. Of course, at 640x480, it would draw stuff off-screen and you couldn't get to it, so there was that, too.

...

There was this guy who was hired on to get SMS working. It wasn't until about four years later that he got around to rolling it out, and when he did, all kinds of things started happening. Right off the bat, they noticed that something like 40 brand new accounts had been added to the domain with full administrator privileges.

The conversation with him and the other "network engineer" went something like this:

"Uh, why did it do that?" (creating those accounts)

"Duh, I dunno."

"And what are the passwords for those accounts?"

"Duh, I dunno."

"You let 40 new admin accounts on and you don't know the passwords?"

They later found out that it does this on purpose: it's actually part of the design. Also, it sets up those accounts with random passwords.

This came after four years of waiting. You'd think it wouldn't be a surprise after that much time had elapsed.

...

Then there was this one time when the first guy was off in traffic court, so the second guy (the SMS one) was there all alone. Well, the primary domain controller just happened to melt down. This took down basically everything, including most of the web pages, since they were mounted over the network from that machine.

All I could do was watch. I remember chatting with some friends and just sharing the latest updates: 15 minutes... 30 minutes... 45 minutes. What was I counting? Downtime. I think it was down for over two hours.

Much later, the best explanation I got from the first guy was that the system had "lost" TCP/IP, so everything based on it like DHCP and WINS had gone with it.

Personally, I think the second guy broke the box while left unattended.

This is someone who used to click on the (phony) [X] button on popup ads and wondered why his browser would get taken to some completely crazy web site instead. It wouldn't be that much of a stretch.

...

At some point, the second one got this notion that the NT servers really needed some disk defragmentation action. Given that I haven't worried about such things since my DOS days, I just scoffed at the entire notion and went about my own business on my Unix boxes.

Still, he pressed on with this project and wound up installing something called DisKeeper on all of the servers. It was supposed to start up, do its thing, and then reboot the box. So one night, it starts up on the primary domain controller (yep, him and that box again), and the box freaked out. Something broke and it didn't come back up.

Just like in the "traffic court" story, this brought down all of the web pages which were hosted there. Of course, it just happened to be the school board meeting night, and they needed to present things which were on that web server...

Yeah. It was another one of those nights.

...

One summer, we were changing over from ISA to PCI network cards. So, one day, I just recompiled my kernel with 3com PCI support, copied it into place, reran LILO to update things, and waited for the hardware to arrive.

When the box of NICs came in, I grabbed one for my machine and went back to my desk. There, I shut down my machine, popped the lid, pulled the old card, dropped in a new one, and buttoned it back up. Then I powered the machine on and got back to work.

Meanwhile, both of the NT guys were fighting with 3com driver disks, rebooting multiple times, and generally had to run around putting out fires to fix everything which broke. I'm guessing they screwed up one or more of their interfaces and may have dropped things which relied on them in a repeat performance of the "traffic court" story.

...

At some point, the boss started using a NT box as a workstation, and it got SMS installed. It generally started hosing over the machine and made it slow to the point of being unusable. His solution was to start logging into his own personal domain since "SMS only comes up when I log into the main one", and "my domain is trusted by the main one".

Okay then. Have fun with that.

...

Not long after SMS rolled out to the client machines used by actual students and teachers. Imagine a bunch of elementary school kids trying to get things done while this updater proceeds to wrap its tentacles around the machine.

One of the teachers was not happy about this and wrote an amazing mail about it. I was on a mailing list she added as a "cc", so I got a copy of it and kept it for posterity.

(Incidentally, this was a public school district, so this is all public stuff. All of our e-mail on those machines was subject to sunshine laws regarding government records.)

She sent this to the SMS guy:

I was concerned that you hadn't responded to my email from this morning, so I called your number and discovered that you're out of town for the week! I am quite frustrated that you did this while away and without answering any of our questions. I now have second graders panicing as it pops up for them. They are also reporting that their computers are frozen, but I've discovered that you can't do anything else while the newuid program is running. By the time I got this across to the class, they'd clicked so many things so many times that some of the computers are freezing up.

I have so many staff members who spend there day restarting and never getting logged in lately, I don't know how many would have even seen the message. Here in the lab by the time I saw the first message, my parent volunteer had already told at least 10 students to cancel it.

Pretty bad, right? Well, the boss responded, and in so doing managed to mention another situation I had forgotten before I started digging into my records for this post.

I apologize for the problems that have frustrated you this week. I try hard to always have one of the engineers on duty, but this week I had to send both to training to learn how to migrate our networks from NT 4.0 to NT 2000. This is crucial to the implementation of the Technology Bond.

Yes, both of these guys were sent to a week-long training class to learn how to go from one version of Windows to another.

...

So, let's talk about the culture of training they had at this place. Every time the smallest thing changed, they would use it as an excuse to go offsite to some place to get "educated" for a week. I never went to any of these things since I thought it was pointless and annoying, and besides, it was only for Windows stuff anyway.

One week, however, things were different. They had this Bay Networks guy in the office for a whole week training them how to use the firewall features of the Bay router software. They wanted to run actual firewall type filtering rules on our big router, and had this trainer brought in. The NT guys invited me to drop in one day to see what was up, so I did.

At some point, I must have asked him about how it would deal with the actual sorts of problems you'd encounter on the Internet, like script kiddies, and mentioned "winnuke". This so-called firewall guy just said "what is that?". Oh. Wow.

Obviously, I had to demonstrate this to him. I grabbed one of their (Windows) laptops and used it to bring in a Windows version of winnuke (yes, such things existed). Then I used it to lob an evil packet or two over at another one of their laptops, and it promptly bluescreened.

They were floored. They had no idea any of this existed. I was just leaning back, enjoying the fun. This was old hat by then, but it was completely off their collective radar.

I should mention that Microsoft had long since released a patch for this "TCP OOB pointer bug" that winnuke exploited. These guys hadn't updated their machines, and so they were vulnerable.

Some firewall specialist, huh?

This memory came back to me in the past couple of days while pondering the general concept of interviewing. It made me wonder if I could still rattle it off in the "unfolding" approach that I did back on that phone call. By unfolding, I mean I don't write the entire thing top to bottom in one pass. Instead, I write just a little thing that does one piece, then insert another, then another, and so on, until it's "done" (as much as software can be done, that is).

I started talking my way through it again just to see how I'd explain it now. It occured to me this might be an interesting topic for a post, and here we are. Let's see if I can do this again right now, having just run through it in a recording (more about that later), but without looking at the code. This will probably have some errors in it, so don't just use it without checking things over first!

So, first, we want a socket. It's AF_INET and SOCK_STREAM and I think there's a third arg that we set to zero. Call this fd, and store it in an int. Okay. Now we want to bind that to a particular address family and port number, so we'll call bind with a that fd, a sockaddr_in, and a socklen_t. The struct sockaddr_in goes above this, obviously, and we need to initialize that to { 0 } to clear out any wackiness. The sin_family is AF_INET, and the sin_port is htons(whatever), because we might have to account for byte ordering differences. "whatever" might as well be 12345 for testing purposes.

Obviously we want to check the retvals from both of these, and if it's less than 0, whine about it on stderr and include the result of strerror(errno) before calling exit(1) to die. If those aren't working, there's nothing much we can do about it.

Now we need to set non-blocking mode on this, so that means fcntl on that fd with some magic value I'd have to look up which means "get". That value gets checked for a negative error condition as usual. Then we call fcntl with the fd, the "set" magic value, and the previous return value | O_NDELAY. This should also have its retval checked for an error.

Next it's time to say that we intend to use this as a listening socket and not an outgoing connection, so we call listen with that fd and some depth for backlogs. This is rubber-chicken waving at its finest, since I'm not sure I've ever run into a situation where I wished I had a higher value for the queue depth. I use 16 for no particular reason as a result. Again, this is checked for an error.

Now we're listening, so it's time to wait for stuff to happen. I choose to use select for the sake of this example even though many alternatives exist. We start a loop which runs forever, and in that loop define a fd_set called rfds. Then we FD_SET the listening fd into rfds and create an int called max_fd and set it to that same value. It'll be useful later.

Then it's time to call select, and the args are max_fd + 1, rfds and a bunch of NULLs for write fds, exception fds and the timeout. If select returns 0, there's nothing to do, so just "continue" back around to run the loop again.

Oh, and before I forget, we want to FD_ZERO rfds before using it.

If select returns -1, then we look at errno. If errno is EAGAIN (or maybe EINTR?), then we probably were bonked by a signal and were interrupted. Either way, it's nonfatal (because "worse is better"), so just continue. Otherwise, it's a problem and we probably need to cut and run since bad things are afoot.

If we're still here in the bottom of the loop, then odds are good that we have a new connection arriving. We use that FD_ISSET macro for the listener fd on rfds, and if true, jump out to a function which will accept the connection, passing the listener fd along. That function just calls accept and hands it the listener fd, a sockaddr_in, and yet another socklen_t. It might fail, so check the return value for negative values and deal with it, but don't die, since it's no problem for us. Just return.

Otherwise, we have a new client on the non-negative fd number which was returned, and we need to start tracking it.

That right there is enough to let multiple clients connect to a server port 12345, but it isn't enough to let any useful work happen. For one thing, we never check for activity on those new file descriptors, so if they talk to us, we'll never know about it. We'll have to add them to the fd_set of rfds before that can happen.

So, okay, back into the code. Let's have a STL map called "clients" from an int to a struct Client, and that struct doesn't have anything in it just yet. It'll be more interesting later on. We create this map right above starting the endless loop where select is called.

Now, we change the accept_new_client function to return a pass/fail bool, so it's false on those error conditions and true if it succeeds. We also make it take an "int*" which will be set with the return value from accept() which is the new client's file descriptor. Back where accept_new_client is called, a new int is created and is handed to that function. The return value of that function is also checked, and if it's true, then we drop a Client into the map using the int as the location. Basically, clients[client_fd] = Client(), more or less.

This means we now know about our clients, and we have to look for activity from them. Up above the select line but below where max_fd is set up, it's time to add a new loop. Create a const_iterator for clients and flip through all of the entries. For each entry, take the first value (which is just the client fd) and FD_SET it into rfds. Also, if it's greater than max_fd, set max_fd to it.

Now select will be looking for activity from our clients, so it's time to deal with that possibility. Down where we test for FD_ISSET on the listener fd, set up another loop to flip through all of the entries in clients just like before. This time, we use FD_ISSET on the client fd, and if true, pass it and the Client struct to a new function called read_from_client.

read_from_client just creates a char[] of some length and calls read with the client fd, that char pointer, and the sizeof that buffer. If it returns a negative value, the client is gone, so return false. If it returns zero, they're also gone, so return false too. What's left is a positive count of bytes, and that's how many bytes in buf are valid. This is where we'd hand the buf off to some kind of parser and say "here you go, you have 'ret' bytes to chew on, oh, and here's the Client struct". For now, however, we don't do much other than saying "read X bytes" and return true.

Back at the call location, we need to check the return value from read_from_client. If it's false, the client needs to be deleted. So let's create a vector of ints called to_delete just before this scan-the-clients loop, and if read_from_client returns false, just "push_back" that fd onto the vector.

Just beyond the client scan loop, now we iterate through to_delete, and for everything in there, call shutdown on that fd with a 'type' value (SHUT_RDWR, I think) and call close, too. Then we 'erase' that entry from the clients map and move on.

Why didn't I just shutdown + close + erase in the actual clients loop? Well, remember that we're iterating on the clients map in that loop, and removing something from that container might invalidate the iterator. Rather than digging around and worrying about whether this is one of those situations where that is unsafe, I just punt the deletions to a second pass and avoid it completely.

I actually recorded myself doing all of this a few minutes ago, but managed to hit ^D one too many times without realizing it during the recording. As a result, it ends after about 8 minutes right after my first test run. At that point, the program only accepts the connection and doesn't deal with tracking clients or anything which follows from that.

Still, it might be an amusing thing to watch. Thrill at the pauses while I jump to a man page in another window! Gaze in wonderment as I somehow manage to go to the exact line which has an error on it... because I'm running the compiler in yet another window!

This one is probably the most "raw" recording I've done yet. Normally I do a bit of prep work and try to figure out what's going to go where before I start. This time I decided to just go for it and let any false starts or dead-ends stay in there. If nothing else, it should give a good feel for the way things jump around at times.

It's a long one, so you might want to watch it at 200% or even 500%. There's also a new feature in the viewer: space will pause and unpause, and all four of your arrow keys will seek through the recording.

Look for "tcpserv" at the bottom of the list. This one is free, so it's already "unlocked". Enjoy!

As nice as this system is, it won't last forever. Wheels are turning in local governments, and the world of public safety radio is going to change yet again. This time, it's going to leave a lot of listeners behind. If I don't find a way to stay ahead of it, that will mean my system as well. Here's why.

Right now, the city system I monitor most of the time uses a trunking scheme called SmartNet. Essentially, one of their channels is nailed up transmitting "control channel" data for a day or so before rotating. The other 7 channels are used for call audio. The radios listen to the control channel for frequency assignments, and when their "talkgroup" is active, they will switch to the right audio channel and will start receiving. Likewise, when you go to transmit, your radio chats with the controller and gets a channel assignment.

The software I run here tracks that control channel so it knows which talkgroup is active on any given audio channel. Without that, there would just be a bunch of recordings but you'd have no idea who was talking without getting really familiar with all of the users. On a big regional system, that would be an impossible task.

The problem is that the city is going to change to a new system. This one is based around a series of standards called Project 25, or just P25. P25 itself isn't particularly new, but it does introduce a degree of complexity which previously hasn't been known in this area. At the moment, none of the local (South Bay) agencies run this kind of stuff.

The new hotness these days is running P25 in the 700 MHz band. Phase 1 of this new system, the Silicon Valley Regional Communication System (SVRCS) will cover Sunnyvale and Santa Clara, and it's supposed to grow as more money is allocated. Eventually, it should cover the entire county and will also tie in with the equivalent systems of the Peninsula, East Bay, and beyond. Ultimately, there will be a radio network where different agencies can talk to each other directly.

Unfortunately, some of the technologies involved are not the easiest thing to monitor. According to the specs, this new system will be "phase 2" P25, and that means the potential for audio channels which use TDMA and manage to cram two digital voice paths down a single 12.5 kHz channel. The days of just having ordinary FM audio are numbered, in other words.

All of my research suggests that only one specific scanner model can follow this kind of audio channel... and it's no longer being produced for some reason. That means if you didn't already have one and can't buy one somewhere (used), there will be no way to listen to the new system. Even if you do manage to find one, it would still be subject to the annoyances of traditional scanners.

Personally, I've been spoiled by my logging system, so going back to a traditional handheld scanner is not something I want to do.

So this is the state of affairs: this new system is coming online later this year, and it's using a scheme which has never been used in this area. I'm going to have to come up with a way to track it, tune it, and decode it in order to continue having a usable site. Otherwise, once they're done with the testing and acceptance and switch over, all of my existing tools will stop working.

I don't intend to let my hard work become useless. Unless they wind up encrypting the traffic, there should be a way to monitor this. I just need to get it going before they finally move away from their current systems.

This should be another interesting summer.

In terms of programming, there's Gtest and Gmock. These two projects are open-source, so they tend to show up in #include directives in external projects, too. The last one I could think of was "GVENT": the SMS short code used for updating your calendar.

I suspect there are others, but I must have forgotten them (or blocked them out, take your pick). There are also a bunch of initialisms or acronyms, some of which are better known than others, but most are not particularly well-known outside the company:

GWS, GWT, GWE, GWO, GSU, GFE, GHR, GDC, GFS, GAR(field), GRADS.

You'll see "gws" in HTTP headers on the outside world, but that's about it. GWT made it to the outside world as open-source, and GFS has starred in a white paper or two. For the others, well, you'll just have to guess what they all mean. Sorry.

As you can see, there's a certain scheme here, where you purposely try to name things to start with a G to give a sense of cohesion. The company's initial gets to start in your project or product's name. You can see that it's "G + mail", and "G + bike" and "G + mock", and so on.

The "G + this" and "G + that" pattern is why this last one stuck in my head. I can't imagine why it was named what it was, but it happened.

The project was named grape... you know, like the things you squash to make juice, ferment to make wine, and dry out to have raisins?

Only... it was in that "G + blah" environment. Given that frame of mind, what does that name look like to you now?

Yeah. I don't get it.

First, start crawling the web for Github usernames. Dig through all of their profile and activity pages and look for links to other users. Keep a note of every single username. Also go to sites like Hacker News and reddit's programming-oriented areas, then try those same usernames on Github if they weren't discovered already. If they turn up more connections, follow them.

Eventually, there should be a giant list of usernames. Now it's time for phase two, which is where the distributed botnet starts hitting https://github.com/(username).keys for every username it already found. github will happily spit out every public key which is currently associated with an account. Did you know that? Before a couple of days ago, I didn't.

I'm pretty sure this is by design. I never wanted my SSH keys, "public" notwithstanding, to be available to the world. There is no reason for it, and of course I see no way to turn it off, so I just removed all of them from the account. It's not like I push anything out there anyway.

Phase three is to set up some kind of honeypot which attracts a bunch of people who are likely to use Github. Invite them to ssh in to your machine to solve your challenge or something like this. Such things have been on HN in recent times. Set up the account to use public key authentication and preload it with all of the keys you collected from phase two.

Phase four involves taking the logs of connections and associating IP addresses with keys, and thus with accounts. Now you have a pretty good idea where that person is or has some kind of account. Further analysis like geolocation is left as an exercise for the creeper who actually tries this.

I'm not a particular fan of their always-visible activity log, either. If I poke around on the site and start following something, there might be a reason I'd rather not have it shared with the world. Maybe I don't want to leak any data about the future directions of my projects.

I find the whole thing rather annoying.

Yes, through the magic of overhead signs, we wind up going south, west, and north all at the same time. Usually you see these things in pairs with a "corner" that makes sense (north and west, south and east, that sort of thing), but the third one here makes it extra-special.

I came up with this little doodle to possibly explain it:

Notice my little mistake up by the "55" - it's a bit of a mind-twister when you think about it, and I put the arrow on the wrong end initially. Oops.

I guess there must be a narrow spot where it made more sense to stack all of these things on the same road instead of letting them go their separate ways. Now I want to find some place where someone managed to get all four cardinal directions on the same stretch. I'd award bonus points if they put huge electric coils in the pavement to screw up any compasses in your car. Call it the "Bermuda Triangle Road".

...

Loosely related to the above pic is this one from the same trip. It does sport a wacky sign, so I guess it fits with my usual collections of strangeness. In this case, "Bar-B-Que" aptly describes what happened.

I'd call that a char-broiled restaurant.

Wasn't there some folk tale about the guy who burned down his house while trying to cook something, and while he lost the house, the food turned out to be amazing? So every time he wanted to make amazing food after that, he built a new house and burned it down?

Doesn't that sound a lot like the software business? Hmmmm! It's sort of like the whole cargo cult thing, only modified for kid consumption.

I can't seem to find any notion of this story online. If you know what I'm talking about, would you please send me a note? Thanks!

I realized the other day that I can actually tell if some of my "lessons" are working by keeping a careful eye on my web server logs. Last month, I wrote a post about writing a really quick hack to read the "protofeed" file I now create for these posts. I recorded all of the steps required to do this and put it up as part of my "/edu/" project.

As it turns out, when people get the first recording working, it shows up as a hit to "/w/protofeed" in my web server logs. It's pretty distinctive, too, since it looks like an ordinary run of curl and isn't something like a web browser or one of the usual web search engine crawlers.

This is pretty cool! It's just so rewarding to see it all coming together. Someone pokes around at the "pfr" lesson recordings, and a few minutes later I start seeing hits to that protofeed file. I know that could only happen if they managed to glue it all together and got things running.

To everyone out there who's playing with this stuff, you are awesome. Thank you for checking it out, and I hope you will enjoy the future recordings I have planned.'

The past couple of recordings have evolved our little "QUERY_STRING" grabber into a nice little library and bounced a bunch of made-up strings off it via the testing suite. It's about ready to be used in an actual CGI program which is called as the handler for a form submission.

That's when actual browsers get involved, and it will start looking more like a web application and less like a bunch of scaffolding. Once we lay down a foundation, it might be surprising just how many things can be built on top of it.

I found a similar annoyance with the whole "scroll, then push control" thing back in 2011. This time, it's the same sort of interaction, but the command key is implicated.

In short, two-fingered trackpad scrolling inertia plus the command key equals huge fonts or tiny fonts. Yep, it seems to have jumped from control to command, and the thing I squashed 18 months ago is back.

This time, the fix seems to be mousewheel.with_meta.action = -1.

I can only assume this happened with Firefox 20, since it seems to have sprung back to life in the last several days. I don't think I started behaving differently for no good reason.

That's what really bothers me about changes like this. For a couple of days, you think you're nuts. Then you have to go digging for changelog references in order to regain your own sanity.

I spotted this one while virtually cruising El Camino Real in Apple Maps to look for anomalies. This one jumped right out at me. There's no Olive Garden in Santa Clara! Trust me, I know this one well. For the longest time, you had to go to Fremont, down to Blossom Hill in San Jose, or up to Palo Alto. There's one in Milpitas now, but there still isn't one in Santa Clara itself.

Still, it got me wondering. How is this possible? I clicked on the entry and this is what I got. Guess what. It's 2515 El Camino Real all right, but it's in Palo Alto! Yes, three towns away. How do they get this wrong?

This sure looks like a Yelp page, but if you go to Yelp, they have it in the right place, as you would expect. Apple is making them look like garbage.

Here's one in a similar vein:

No, no, no! There is no Jack in the Box there in Santa Clara. There are two on El Camino, yes, but one of them is down by Nobili and the other is over by Lafayette. That particular spot is actually La Paloma, a real sit-down Mexican restaurant.

What happened there? Oh, that's easy. Look:

Just like with the Olive Garden, they took a Palo Alto address and transplanted it down here. Do they have toxic waste in the drinking water at Apple Maps HQ?

Moving on...

Big Siky Trail? Try "Big Sky Trail". Are they doing manual data entry?

"Domn it! I used Apple Maps again!"

They think the Caltrain station is way out there for some reason. See that thin line running along Central? Those are the tracks. You'd think the station would be, you know, on the railroad tracks?

They even manage to second-guess themselves. Check out what you see when you scroll the view just a touch up and to the left:

Yep, a second icon, about where it should be.

This one is comparatively minor but it's still weird. The store is set way back. They got the Best Buy icon more or less correct. Chipotle, however, is much closer to Charleston.

Is there anyone who doesn't know "1600 Amphitheatre Pkwy" for That Place? There's only one center of the darkness, and they missed it.

Chase is that big building facing into the intersection. It has a lot of unusual adornments which might make you think it's a museum at first glance. They put it in the wrong block.

The Pruneyard in Campbell is a big, busy, crowded place. They would have you believe Outback is way up there right next to Bascom. It's not. It's actually on a separate pad site way around the corner, practically not even in the shopping center any more. You'd be searching for the restaurant for a very long time if you relied on this map.

Notice how far out I had to zoom to fit both the icon and the actual building on the same page. See those little specks? Those are cars. You can figure out the scale from that.

Say hello to the "Normandy" area. There's a gas station or two, a Burger King, a bar, a donut joint, a bunch of apartments, and even some kind of school uniform place, but a whole university? No. SJSU has a massive footprint in downtown San Jose, as you would expect. It isn't on the SJ-SC border.

I can take a guess at what happened. They apparently have a "1 Washington Square" address... in San Jose. The address shown on this map would be approximately "1 Washington Street"... in Santa Clara. Yes, it turns into Bascom Ave when you cross into San Jose.

Dumb.

You'd better search "one more" time, since there's no restaurant here. This seems to be a spot used by the people who make SJC security nervous when they shoot videos of planes coming and going. I suspect this is actually inside the terminal and it picked up a really bad placement courtesy of an "Airport Blvd" address.

Unless you're looking for worms or bugs, don't go here for food.

There's no DMV in Milpitas. From what I can tell, there never has been one. This address apparently got tangled up with one of those scuzzball web sites which puts up a bunch of plausible-looking content to drag people in and get their eyeballs for ad impressions. How it graduated from that to an actual full-on DMV blip on the map is anyone's guess.

One amusing thing: searching for this address on various search engines occasionally turns up things about "RMV" registration. RMV? That's Massachusetts! Hello, the other side of the country called, and they want their office back.

That's the bottom of my stack of snark... for now. If they keep using these horrible data sources, I'm sure I'll find more in the future.

There's an add-on which will bring it back, but it only binds to shift-ESC, so it's no longer a simple "jab at the corner of the keyboard" move, but instead requires a modifier key, too.

I can't even remember how far back the "ESC to stop animating garbage" goes back in the world of web browsers. I used to purposely binary-mangle my Netscape binaries way back in the day to make them unable to pick up on the extensions which specified looping. Then they added ESC-to-disable at some point, and life was good. I could watch animated GIFs if I wanted, and I could stop the annoying ones too.

Now, this is the state of affairs. How stupid.

If you want to see the logic which went into this decision, be sure to check out the bugs: #614304 and #824248.

Have fun!

The project's name was "Rack: a Ruby Webserver Interface". I clicked through to their web page and was rewarded with this, and a wee bit of shell-shock:

It's not the title or the cute little image. Oh no. It's their font choice. I had a flashback to days of old where umbrellas were opened indoors and things like Project Darkness fought "the man".

Want to know why? Here's why.

That picture is from 2004, showing the post-dotcom and pre-IPO styling. Same word, same font, same color. Is it any surprise I shuddered?

...

As long as I'm going on about graphic design and this particular company, I might as well share this little gem. I can't figure out if they were going for more of a Soviet thing or a North Korean thing.

At the time, there was an entire division of the company we called the "fourth reich" because they were so ridiculous in how they did things: tons of useless paperwork and no attention to technical proficiency. Then, one day, this thing popped up on the wall. It was too perfect to ignore so I took this picture.

...

I should point out that we moved into a half-baked building. Not even the proxcard readers worked completely for the first couple of days. Then, when they *did* start working, there was no warning. The first time some of us entered the building after they switched things on, one of my friends grabbed the door and pulled it, just like we had been doing for several days before that point. It opened... because he had just pulled it off the (not very strong) magnet. It made a nice *BOINGGGGG* noise as it popped loose, and then the receptionist growled at us and said "don't do that!" as if he did it on purpose.

If you go through a door a bunch of times and the card reader does nothing, pretty soon you're just going to grab the handle and pull on it like a normal human being. When they turn on the card reader and get the lock working, there's going to be a period of adjustment. Yelling at the people who had no idea it had just "gone hot" accomplishes nothing.

I have a suggestion to anyone who finds themselves in this sort of situation and yet wants an answer to their question. All you have to do is broaden the scope a little.

Which team are you on?

That's a great question. If you asked me that over the years, you would have gotten something like "it doesn't have a public name, but it's the big account system you log into", or "it also doesn't have a public name, but it's the scary-big database which stores all of your prefs". After that, there was the "kernel qualification automation team", and then finally "yet another project without a public name which acts as a nanny for all of the machines in production".

If it's a really small company, you might hear something like "well, we're all on the same team", and that in itself is potentially useful information. Likewise, if the interviewer can't answer because they don't know or aren't allowed to give even a sanitized description of their job, that might tell you about the environment at that company.

Let's say you still don't trust that your interviewer is technical. I don't know why you'd feel the need to suss that out during a phone call which is all about you, but let's say you go there anyway. You can still get an answer without coming off as some kind of sexist ramrod.

What's a typical day like for you on that team?

Assuming they answer it honestly, then you might be able to learn quite a few things about what they do, their credentials, and whether you want to be a part of it. Imagine a description of a day which went like this:

Well, today, I guess I got to the office around 10... late enough to where the carpool lanes are open to everyone and the traffic has died down, but still early enough to actually get a parking spot by my building. I pushed out the new version of our storage software to a couple of locations and checked on how it was doing at the places where I pushed it yesterday. Then I updated the push schedule for next week with the details.

I guess after that it was time for lunch, then I scooted off to make this call to you. After this, I'm going to key in my reactions to our applicant tracking system. I'll probably wind up working on my project to add better logging to our production toolbox after that. I may or may not grab dinner at the office, and then I'll probably go home by 7:30 or 8 or so.

Here are a few things you can learn from this ...

One, your interviewer drives to work alone and this implies she does not take the company shuttle for some reason. Two, traffic in the region can be a pain (that's not really news, though). Three, parking anywhere close to the office can be a problem. Four, she does pushes to production. Five, she does QA checks on those pushes. Six, there's a "push schedule", and she creates or maintains it, or perhaps just has write access to it.

... and that's just the first paragraph.

If that doesn't strike you as the life of a typical engineer, then I don't think anything will.

According to this screen shot, you can find the Santa Clara / Great America ACE/Amtrak train station right there across from SJC. It's not there. It's not even close. It's about two miles north of there, sitting under the Tasman bridge. It's not even on that side of the street or tracks!

This puts the new 49ers Stadium at the entrance to the Great America theme park instead of over by the training center where it is in reality. There's also a good-sized stream running through this area, between the theme park and the stadium site, but you'd never know it from this map.

Please don't drive this way on this little connector road at SJC airport. It won't be a pleasant experience. All of the traffic in that spot is headed one direction: up and to the left.

I have no idea if there ever was a "College Park School", but there certainly isn't anything of the sort at that spot now. This is part of the former "Coleman Loop" neighborhood which was in that general area. According to my research, they started clearing it back in 1974.

Here's proof that nothing exists on this spot today besides a field:

See, it's a couple of ramps for 880 and a bunch of dirt.

Moving on...

CHERVON? Are they doing manual data entry? Where's the spell checker? And what's with the ALL CAPS? Besides that, both the gas station and Lowe's are completely misplaced.

Togo's is across the street. The parking lot for Wells Fargo is on the indicated spot, and has been since at least 1962.

You think they'd all be spelled the same way, but no.

According to the city web site, this was renamed Marsalli Park in 1998. 15 years ago!

The topmost rail spur by Hitachi was pulled out of the street within the last two years, and it hasn't been a viable crossing any time I can remember. It was just a bunch of bumpy rails in the middle of the street with nothing on either side. The other two haven't been there even longer than that. You can't even see evidence of a crossing with the other two. Who knows how long they've been gone.

It's "St. Justin Parish" (or School), not "Justens", and Popeye's is way over on the corner.

The library is that big building. It isn't in the street.

These two are way off. They're shown practically on top of a Taco Bell (which is on the corner) when they are set back from both streets quite a ways. Sumiya Yakitori is somewhere set back in there, too.

I have more... many many more... but that's enough for now.

Hacker Monthly issue 35 is out, and it includes a reprint of my "stupid hour" post from November.

If you've ever wanted to read some of my stuff in dead tree format, now's your chance. Oh, and the rest of the issue looks interesting, too. :-)

Sometimes, there were bugs in the actual server code, and they could be used to really make life miserable for other people. I saw this one happen back in April of 1996 and never really understood it at the time. Now, I can look back and start making sense of the whole thing.

A rumor started going around that someone had found a bug in the "channel key" code. This was something you could set on your channel which would bar access to people unless they specified that same key when they attempted to join. Without that key, they'd only get an error message. I figured it was the usual "embedded \0" C madness and started playing around with it, but never really got anywhere with that.

Some time later, a few of us found the actual patch which had been going around and tried to learn from it. There was one interesting part in the code which was in the neighborhood. Check it out and see if this makes you go "hmm":

		u_char  *s;
 
		for (s = (u_char *)*parv; *s; s++)
			*s &= 0x7f;

In this case, "parv" is the key supplied by the user right after it's been through a sanitizer which removes things like spaces and other unsavory characters. In case C isn't your thing, I'll explain what this loop is doing.

This loop just starts at the first character of "parv", and looks at each one until it hits the end (\0). While it's doing this, it replaces the current character with one which has the high-order bit cleared. It's doing a bitwise AND with 0x7f, which is just 01111111, so something like 10000001 would turn into 00000001. No big deal, right?

Well, actually, it was. Imagine what would happen if you passed it a string like this: "foo(141)(138)blah", where (141) and (138) are characters with those literal values. First it would pass through the sanitizer above this block, and it wouldn't find any spaces or carriage returns or anything like that, so it would then drop into this for loop. In this loop, it would strip the high bit off the 141 and 138, turning them into 13 (141-128) and 10 (138-128), respectively.

What's a 13 and a 10? Oh, just a carriage return and a line feed. So now you have this string which is "foo\r\nblah". What happens next is the server will push this mode string out to all of the connections where this channel exists. That means all of the locally-connected clients (people), and all of the servers too. It looks something like this:

nick!user@host.name MODE #channel +k foo
blah

Got that? It's an ASCII protocol, so it uses chars like CR and LF to split up commands. You just managed to escape from the MODE line and onto your own command line. It's the IRC equivalent of blasting 2600 Hz down the line back when that meant something. You now have control of the content in there.

What can you do at this point? Almost anything. Maybe you want to kill someone's connection, so you send "foo(141)(138)KILL victim :ha ha", and it goes out like this:

nick!user@host.name MODE #channel +k foo
KILL victim :ha ha

Lots of bad stuff started happening. Not surprisingly, a bunch of people tore down their servers for immediate patching and restarted a few minutes later.

What was funny is that the patch added a warning which would go to the server console, and it would say if someone was trying to abuse the "+k bug". Then, if you were dumb enough to try this on a patched server, the operators could see you and ban you for being annoying.

It was pretty crazy. It's also another example of what can happen when you use in-band signaling and have ambiguity between your field or record separators and the actual data which is provided by users. It screws up protocols like this, it makes life interesting for the web (with all of the angle-bracket and ampersand escaping we have to do), and it throws monkey wrenches into Apache log parsers.

For more on this fun little problem, check out the original post from 17 (!) years ago.

The first one is a collection of useful things you can do with Apache .htaccess files. If you ever wanted to have a file in your document root which executed as CGI instead of having to bury it in your ScriptAlias directories (with relatively ugly URLs like "/cgi-bin/..."), it's in here. If you want to play around with denying the world and allowing some back in by hostname (or IP address), that's here too.

Finally, I did a quick demo where I rigged some rewrite magic so a hit to /abcd actually internally hits /foo?module=abcd. This is useful for minimizing ugly URLs which are too closely bound to whatever software you happen to be running today. It can also be handy if you need to change to something else without breaking your well-known URLs.

The second lesson continues the query string code from Friday. This time, I take the "all in one .cc file" situation and rearrange it into an actual class. Then I hook it up with GTest and GMock and hit it with some unit tests. It's a great way to see how a simple little blob of code can start evolving into a proper library.

There's still the matter of dealing with %nn encoding and actually storing this data in a meaningful way. Keep an eye out for future updates.

Back when Buzz still existed, it had a little anomaly relating to how it handled certain special characters. I set up a search to maximize the snark value.

The whole " thing is where the "s should have been.

...

I used to distribute the "Testing on the Toilet" weekly to the women's bathrooms because there was a serious shortage of women engineers to do that sort of thing. It meant a weekly trip to every stall in every bathroom on the second floor of my building. I think it was 19 holders in all.

While making these trips, sometimes I'd find other things. At some point, someone decided to put up these vapid "Learning on the Loo" weekly posts alongside TOTT. They sported all sorts of things, basically advising you how to succeed in a Dilbert-esque company. I happened to catch one of these which could be parsed the wrong way.

"Keeping Active at Work = Reduced Stress + Productivity".

To get some idea of the alternate interpretation I had in mind, just imagine if English used parentheses the way programming languages do. You might parse it like this:

"Keeping Active at Work = Reduced ( Stress + Productivity )".

... instead of the probable intent of the author:

"Keeping Active at Work = ( Reduced Stress ) + Productivity".

Reduced productivity? That's some advice.

...

Finally, it seems I'm always tripping over something strange in one web site or another.

This exists in the filesystem. Okay yumsugar, I believe you.

I wrote a lot of custom code to make those lists render nicely as described in my earlier post. What I haven't talked about yet is how all of those songs were extracted from my CDs in the first place.

Early on, none of the CD-ROM drives in my life would do the "CDDA" digital extraction stuff for audio. The only way to get music off the disc and into the computer was to start it playing somewhere, loop it back, and then record it as a WAV file. Or, if I'm talking way back here, as a VOC file (Sound Blaster Pro days!). I never really used that since it tended to sound poor and was a ton of work. Also, disk space was at a premium in those days.

By the time I got a drive which could do reasonable audio extraction, the "sweet spot" hard drives had grown to being about 2 GB, and so I had a little more breathing room. Now I could drop in a disc and run something like cdparanoia and it would give me a directory full of WAV files. Then I'd manually run l3enc on all of them, and rename the resulting files to something meaningful. Then I'd type in all of the album info, track names, and all of that, and shove the whole thing into place.

Finally, I'd run my generator and the new music would appear. It was manual, annoying, and slow. It needed to be improved. My first change to all of this was writing something which would wrap the ripper and encoder and would keep them busy. Then I added something to make it ask me the artist name, title, and date. Then it would scan for the number of tracks and dropped me into an editor to supply the track names. Once satisfied with my data entry, it would go off and do the work for me, including making reasonable filenames.

I'd usually parallelize my work at this point by plopping the CD and booklet onto my scanner to get images for the library. As long as I had the thing right there on my desk, it was the perfect time to get it scanned into the computer. Those tasks would normally wrap up around the same time, and I'd just put the CD away and call it done.

That was much better than the days of running all of this by hand, but it still needed improvement. I knew about things like the CDDB from the past, and it had evolved to become the FreeDB project. Rather than trying to write Yet Another Client for that spec, I just downloaded one that would scan a disc and spit out the raw data. Then I'd just parse it myself.

I wound up with more than I bargained for. The format used by these databases is an utter disaster. If you've already read rants about this, or tried it yourself, you know what I'm talking about. For everyone else, here's the story.

Some of the files start with "# xmcd". I call them "normal". Others start with a line of nothing but "#" (pound, hash, octothorpe, gate, whatever you want to call it) characters, and I call those "multi". They are treated completely differently.

A "normal" file might start out like this:

# xmcd CD database file generated by Grip 3.3.1
# 
# Track frame offsets:
#       150
#       24430
#       45347

Yes, that's right, it's using # as comments, but it also seems to be storing potentially useful information in those comments. With those frame offsets, you could get some idea of how long a song is, given that "Red Book" audio frames are 1/75th of a second, or about 13 ms. If you want that stuff, you have to write a parser which will ignore the fact it seems to be a comment and will go spelunking for it.

Later on, you get this:

# Disc length: 3152 seconds

Again, that's potentially useful information. From there, it starts listing actual data about the CD, like the title, year, and track names:

DISCID=930c4e0b
DTITLE=Sophie B. Hawkins / Tongues And Tails
DYEAR=1992
DGENRE=Rock
TTITLE0=Damn I Wish I Was Your Lover
TTITLE1=California Here I Come

No big deal, right? So, now have a look at the "multi" format. This is what you get when the database has more than one match for the disc fingerprint you submitted.

Here's the beginning of one such file (truncated a bit to fit here):

########################################################
0 data a80c840b Billy Joel / Greatest Hits Volume I und 
########################################################
# xmcd
 #
 # Track frame offsets:
 #      150
 #      25415

Did you notice the "data in a comment" part is actually indented one space? If you wrote your parser to handle the earlier situation, you now get to extend it to deal with this as well. It actually indents the entire block of data for the disc, including the DISCID, DTITLE, and everything else like what is shown above for my Sophie B. Hawkins CD.

Then, it jumps back to column zero for the next match:

#######################################################
1 misc a80c840b Billy Joel / Greatest Hits Volume I &
#######################################################
# xmcd
 # 
 # Track frame offsets:
 #     150
 #     25415

It's not some incredible coincidence of two discs that happened to hash to the same fingerprint. Nope, it's the same disc, entered two different ways. I had to add code to my program to show all of the matches to let me pick the one I wanted to use as a base. Still, I couldn't take that data directly, since it was frequently screwed up in ways too numerous to mention.

I took my existing program and just rigged it to use the CDDB type data as defaults. If it looked reasonable, I could just hit ENTER and take it as-is. Otherwise, it would drop me into my editor and I got to clean it up and set things right.

I think this freedb parser is the first code for which I ever wrote unit tests outside of work. In that directory, I have a couple of files which represent real returns from a database lookup. My test code just sends them through the parser to make sure it gets reasonable values out.

It's all supremely evil code, and yet it was still better than the alternative: manual data entry for everything.

Long version:

Okay, it's been long enough, and it's time to start my series of small lessons for building web applications in C++ from (almost) scratch. The point here is not to make something which is going to be a real site, but rather to show what sorts of things happen along the way, and ultimately end with something which can be used to do real work.

You have to build up to writing nice code where everything is clean and laid out just so. With that in mind, I'm going to approach some problems from a relatively low level and work my way up.

The first task is to handle a HTTP query string when run as a CGI program. Whenever you see a URL which is like /foo/bar?a=1&b=2, that "a=1&b=2" part is the query string. This will show up in the environment as a variable called QUERY_STRING. You need to fetch it and parse it into something useful -- preferably something which will allow direct access to variables by name: "a", "b", and so on.

A web framework typically does this sort of thing for you, but we're operating at a lower level for the sake of exploration and learning here. That means we get to handle all of it directly.

In this lesson, I demonstrate reading it from the environment and then slicing it up with a simple state machine.

You should also tune in for future installments where I'll split things up for testing and general cleanliness, and then write those tests and run them in coverage mode. Then it'll be quite obvious how many paths have been tested and which ones still need some work.

Ready? Okay? Let's go!

It's cheaper than a trip to your favorite coffee shop.

It seems his program blew up when running on certain devices. It was hitting a memory limit which existed on some of the smaller models, and the kernel was shutting him down. He managed to instrument it to see where the memory would spike and then slowly work its way back down to prior levels. It was concurrent with a fetch from his backend server.

He started poking around. This fetch was essentially a call into some system library which said "hey, go get this URL for me". Unfortunately, that library decided to go off and bring in the whole thing as fast as it could. All of this happened inside his app's address space, so the buffer grew and grew. Then it would hit the limit and the OOM-killer would go off. Oh, the embarrassment.

His fix involved twiddling some knobs which made it not try to chew nearly as much memory with buffers. This got rid of the spike, and the program no longer died on that particular kind of device. This made everyone very happy, and there was much rejoicing.

Upon hearing this, I realized how strange it was. He's running on a device which is limited in terms of CPU speed, RAM, and even flash storage. There's also a battery to worry about, and to make matters even more interesting, you might find yourself running on a slow cellular link from time to time. On this system where you have to be very careful about using the minimum set of resources at all times, the APIs are thick and heavy. They tend to put you at a great distance from the things like TCP sockets way down under everything.

Then, at the same time, there's the state of programming for desktop machines. I have this stupidly large Linux box here for running my software defined radio stuff. It has something like an 8-way CPU, several gigabytes of memory, hundreds of gigabytes of disk space, and an always-on Ethernet connection to a pretty fast cable modem pipe. It also has all the juice it can need by virtue of being plugged into the wall at all times. There is no battery to worry about.

On this machine, you really don't have to worry about saving CPU cycles, memory, disk space, bandwidth, or battery runtime. You can go all-out and just waste things outright if you want. So, what sort of programming happens on this box? Well, most of it is plain old C and C++, or pretty close to the bare metal. There aren't multiple layers of APIs between me and a network socket, for instance.

Isn't that strange? The itty bitty "boxes" with limited resources have these relatively fat frameworks happening on them while the monster boxes have people running extremely close to the bare metal.

Sure, there are reasons for why things are like this, but it's always interesting to take a step or two back now and then. Sometimes, you'll see a situation which really might not make sense to an outsider. Seriously. Try explaining the "slow machine + fat framework" and "fast machine + lean code" thing to a non-programmer. They'll probably think we're all nuts.

Who knows. Maybe they're right.

The kernel hackers who were affiliated with one of my former teams would be a good example of this. They were from all over the world. I met people with all sorts of interesting backgrounds while working there.

So you're reading about something this group did, and the article includes a line like this:

Fortunately, the American kernel programmers knew how to write quality code.

I'd hope you'd look at that and think "gee, that's funny". Why did the author need to single out the Americans? After all, there are lots of people from other places who also work on the project, and they do good work too. Is this article trying to say that the others don't write quality code, even though they do?

I'd expect most people to look at that and think it was anomalous. You could hope that it came about as a sentence which changed directions halfway between the author's brain and the keyboard. That happens to me from time to time, and you'll see weird words floating around which don't belong there. I get reports and I fix them -- thanks, Michiel.

Anyway, if you assume it's an oversight or error, then it shouldn't be a big deal to raise a question about it in a discussion forum.

Just the American kernel programmers? Why not all of them?

Imagine if you said that in a community of programmers and started receiving responses from people who were clearly not agreeing with you. I'm talking about things like this:

Given the time period in question, I think it's quite likely that all the people working on this project were Americans.

Or, something like this:

"American" also means "citizen of the world".

Or, this one, which is much longer:

As far as I know, foreigners didn't have anything to do with quality code, or designing the kernel itself. Therefore, yes, "Americans" is the correct word to use.

[...]

I'm sorry you think it is more productive to call out perceived slights to your nationality, rather than doing something awesome. I guess it's a lot easier kvetching than, you know, actually doing something.

But wait, there's more.

Could somebody let me know if this is going to be a thing on every thread in this forum? Because if it is, I'm going back to reading it through RSS.

Is it so much to ask to not be forgotten once in a while? Is it even possible to do that without receiving a disproportional response of hate?

Apparently not.

I will attempt to relate some of these things. I can only speak about the things which have happened to me, in the assumption they happen to others. However, you should actually talk to other women before attempting to apply these stories to everyone.

...

If you perform an interview for a technical position, you run the risk of being assumed to be someone in HR. I did nearly 100 interviews, mostly over the phone, over a 4.5 year period at the big G, and on multiple occasions had a candidate question my position. When this happens the first time, you sort of cock your head sideways and think, wait, did I really hear that? Did he really ask that? Maybe they ask that of everyone.

So, then, afterward, I'd check in with some of the guys around me. These were both teammates and people who just happened to sit nearby. I trusted they would give me honest answers to my questions, particularly if I didn't seem to have an agenda or otherwise wasn't "on a roll" when I brought it up.

I'd say, hey, did you ever have a candidate ask if you were an engineer? Or, did you ever have a candidate ask if you were in SRE? Or, did they ever think this wasn't going to be a technical call?

None of them ever had that happen to them.

...

For my first two years there, I wore a pager and was on-call for a team which supported a pair of services and later split to supporting just one of the two. The service I kept up had a whole bunch of personally-identifiable information (PII) about users in it, and so it had been constructed with an extra degree of paranoia.

It ran on its own machines, had them rolled up into its own "laser-mounted-on-the-head" clusters, ran its own storage cells using the multi-disk machines, and then ran its own database on top of those storage cells. All of that software came from other teams. We built, configured, and maintained our own instances of those things in order to actually run our service on top.

This was all done in the name of security, since there was a time when merely becoming an engineer would otherwise grant you full access to the kingdom. This particular service was isolated in numerous ways to keep that data safe. It meant duplicating a bunch of things which were normally run by their own dedicated teams. It also meant that I got to touch the "full stack" and not just my little app on top of it.

In subsequent years, I left that behind and went on to a development environment where we were testing kernels and other things. There were some people on the larger team who wanted to use some of these storage cells to keep copies of their test data, because it was rather large. They also decided the amount of quota required (which came out of some budget somewhere) was excessive, and so they wanted to do something very strange.

Normally, the storage cells wrote things as "r=3", that is, for every chunk of data you write to it, there are at least two other copies somewhere else in that cell. If it drops below that for whatever reason, the other two replicas spring to life and start "cloning" it to other machines in the cell to bring it back up to that level.

This means if you obtained 1000 GB of quota, you really could only store about 333 GB of user data, since the other two copies consumed the rest of it. These guys saw this as wasteful and so decided they were going to purposely set their data to "r=2".

I told them this was a horrible idea. The system was not designed to be used that way. In fact, if you stored your data that way, and something happened to it, the people who ran the storage service would not help you. They actually had a big warning on their team web site which basically said "we will not attempt to recover missing chunks which are less than r=3".

I further told them that if space was an issue and if their data was immutable, there was something you could do to mark the file as "archived" which would do some magic encoding to it. It would no longer consume 3x the space after that happened, but it would still have the same availability. It wouldn't quite shrink things down to merely 2x, but it was a good middle ground.

They didn't listen to me.

Some time passed, and I found out they had been talking to some guys somewhere else in the company. Those guys told them the same thing: r=2 is a bad idea, so use the magic archival encoding setting and the cell will give you back some of that space.

That's when they came back and said "okay, we're going to use the archival setting".

I fumed. I had told them this days before. I even qualified it and said that I had run the storage cell software in question and knew how it performed. Heck, I even built some of them from scratch on bare machines once in a while, and that involved a whole bunch of grungy stuff most of the maintenance type SREs never do!

They didn't believe me. They had to hear it from someone else.

Dismiss it as just a fluke if you want. Say it's not because of who I am if you want. It still happened. And it happens. And I hear it from other women in the industry, too: an idea or concept from their mouths is ignored until it's repeated by someone else who happens to be a guy. Meanwhile, an idea from a guy in the same environment is at least considered the first time out.

Obviously there are counterexamples. I'm just saying that this does happen from time to time. Does it happen to you?

...

Not being heard is not always one of these "online interactions are weird" things. Sometimes it literally means not being heard, as in, the vibrations in the air which we call sound.

It's a Monday morning in August of a few years back, and that means it's time for the weekly production meeting. There are a bunch of people sitting around a big table. It's maybe ten feet long and five or six feet across. The room is barely bigger than that, with just enough room to fit the table, chairs, and space for the door to swing, plus screens for the projectors at the far end.

It's not a big room, in other words.

Everyone is seated, and some topic is raised. One of the guys is saying something about how type 2 traffic is lossy over the backbone. Another one is talking about how it doesn't matter. A third mentions maybe there could be a workaround.

I notice this: hey, wait, type 2 traffic? We don't use type two traffic.

I go "uh... wait." Nothing happens.

They keep talking. They're going more quickly now, and are a bit louder than usual. One of them has come up with something they can do about this so-called packet loss for type 2 traffic, and the other one doesn't like it. Never mind the fact that we don't use type 2. I need to stop them before they start making decisions for something that won't even *DO* anything.

"Guys? Hold on there." I probably leaned in and gestured with an open hand (palm up, thumb out, like "what?") while doing this. It should have been obvious that I had something to add.

Nothing. They're still going.

Yap, yap, yap, back and forth. Oh, we can do this, no that sucks, no, we'll do that instead. But then we have to do this. Yes, their hacks are going to lead to bigger problems. This has to be stopped.

I'm practically waving my arms around now. Still, nothing. WTF?

They're still going.

It's been a crappy couple of weeks for me. I've been having a miserable time at this job, with the oncall and treatment by the others on my team in general. I'm over it. I raise my voice and speak up, making sure they notice.

"PUT DOWN THE CRACK PIPES!"

Okay, that worked. They're now all looking directly at me, and they're shocked.

"We don't even use type 2 traffic for replication, so none of this matters. We're at type 1 because we are in the serving path for a type 1 service and have to be as reliable as they are. The only time we use anything else is for the metadata, which travels in the swamp, and that's because of that bug I tracked down and reported, and they're sending us a fix in the next release."

They're not happy.

There's an awkward silence.

Finally, the boss or someone else says something like "okay, moving on..." and they go on to the next agenda item. Nobody makes eye contact with me.

I've stopped yet another half-assed idea from going into production, but at what cost? I find that out immediately after the meeting when my boss pulls me aside.

"This is materially affecting your perf score."

Translation: the number which is used for calculating my raises, bonuses, stock options, promotions, and the like, is lower than it otherwise would be because of the way I dealt with the rest of the team.

So, basically, my options are: "be ignored and have bad things happen", or "get noticed and have bad things happen". The middle ground, if there is one, is impossibly thin. The slightest misstep will put you onto one or the other, and then you're in trouble.

Kindles and Kindle content are currently unavailable in Indonesia.

We value our international customers and hope to make Kindle and related content available in more locations in the future.

Thanks for your interest in Amazon KDP.

While this was going on, I decided to work around the problem myself. I'm already set up with PayPal to clear credit cards, so the solution was obvious.

So then, if you want to buy a copy of The Bozo Loop but can't get it through the usual vendors (or don't want to, because you don't like them, which I totally understand), you can now buy it straight from me.

Bozo Loop, direct

I am contractually obligated to keep the price I choose the same everywhere, so it's $7.49 in all three places: Amazon, B&N, and right here.

I have both the .mobi file originally uploaded to Amazon and the .epub file I created for the Nook available. None of them have DRM enabled, just like the downloads you'd get from those online stores.

Orders are fulfilled personally via e-mail attachment.

So, if you're looking to catch up on 2011, or just want to support my writing, please check it out.

If PayPal isn't your thing, well, I've been working on something on that front, too, but it's not quite there yet. Hint, hint.

I've received some illuminating replies to my half-baked idea about ISPs which only give local service and transit is your problem. Victor wrote in to say that Russia has "Home Networks" where you have Ethernet service to your local community, but have to resort to PPPoE to get beyond that. This external service is metered, throttled, or both. It seems people sometimes set up local FTP servers to make the most of the fat local bandwidth. Pretty cool!

I also got a note telling me about an ISP in Wellington (NZ) which essentially did this back around 2005. They hooked you up to the interexchange point, and it was up to you to get out from there. Apparently prices weren't that bad, either - 100 Mbps service locally for NZ$130/month. That's pretty amazing, considering I can't get that kind of bandwidth to anywhere from where I live, and you can't throw a rock without hitting a co-lo around here.

Do I need that kind of bandwidth? Nope, absolutely not. I just want some way to keep Comcast on their toes. These guys usually don't start caring until someone else comes along and threatens their comfortable resting places.

This brings me to another half-baked and sort-of evil idea I had about 10 years ago. I was down in Corpus Christi (Texas) to spend a few days on the coast. At the time, the city had been installing wireless access points all over the place.

Everywhere you looked, they had one of these boxes with two antennas sticking up. A quick scan with a laptop showed their "CCTEXAS" network with great signal coverage all over. It wasn't allowing much of the way of public access at the time, but I had an idea for how it could be used for great evil.

Imagine they're doing the typical "walled garden" thing, where you can associate with the wireless network but can't get out to the Internet. If they did it the relatively simple way I built my own wireless gateways, then it becomes possible to sling packets around on the wireless side of things. Two stations can communicate across that "fabric" without ever going out to the Internet.

Imagine if they further did some kind of bridging craziness such that a packet would magically find its way from one WAP to another across their city backbone if necessary. This might happen if they made it one single IP network for whatever reason. We used to do that with the school district wireless network, but the biggest setup was perhaps 3 WAPs. You could walk from one end of the building to another without having your IP address change or connections drop.

In our case, nobody really used that "walkabout" feature, but it was useful in fringe spots where both networks are sort-of visible but it varies, like when kids are moving around in a classroom and occasionally cause the signal from one side to drop out.

In any case, imagine this massively bridged network taken to extremes, and people looking to exchange data covertly. They could grab a laptop, use a boot CD, make up a MAC address, and use a yagi to plink a WAP somewhere relatively far away. The other person would do the same, and now they'd be using the city wireless to do their evil without ever leaving a trace.

I would hope that networks would now be designed to not let this happen, and indeed, not let "station to station" communications occur without a good reason, but my experience suggests otherwise.

Then again, perhaps this has already happened somewhere. It might be interesting to try it in a few places and see just how far things will go.

Right now, when you connect to some random consumer ISP, like a cable or DSL provider, it's understood that you'll get IP connectivity to the rest of the world. If there's some network you can't reach, chances are there's some temporary problem and it's not intentional. Sometimes, these guys play "chicken" with each other over peering agreements, but it doesn't seem to affect ordinary home users that often.

As a result, it doesn't really matter which networks are between you and the rest of the world. You effectively have a free pass across enough of them to where you can get things done. If your ISP hasn't arranged some deal, then their provider has, or the other side has worked something out, or whatever. You get the idea.

Anyway, what I imagined this morning was a world where ordinary end users did not automatically have transit across the various backbone type providers. For instance, you might be connected to Comcast and you could theoretically reach other people who also are on Comcast, but that wouldn't be very interesting. Your favorite cat picture sites are hooked up to some other providers and you need to cross their networks to get there.

So, in this topsy-turvy world, end users would have to work out their own connectivity with a transit provider. Their home ISP -- that is, the company which actually runs the local fiber or copper -- would only take them as far as some kind of local "termination point" for that provider. It would be up to you to make use of it.

This got me thinking about how you might design such a crazy system. One way would be to have the transit providers run their own VPN servers, and either route them (and nothing else), or just park a bunch of those things every place they peer with a "last mile" provider. The end user would get their local IP from whoever, but they'd then tunnel through that to a VPN concentrator which was connected to their local IP provider. From there, they'd be able to cross the transit provider's network and go out to the rest of the world.

Again, to be clear, nothing about this seems like a good idea.

Still, doesn't this sound a little like the way long distance dialing works in the US, particularly with "dialaround"? Assuming normal old-school POTS, you get a dial tone from a local provider. You can talk to other people who are also attached to that same provider in the same area. However, if you need to go beyond that local area (even if they happen to be on the same provider, thanks to LATA boundaries), you need an IXC to get there.

This leads to the whole "PIC" thing, where you have a default provider for your "1 plus" calls, so when you dial out, it automatically gets relayed to them by your local telco and they bill you later. Or, you dial some kind of access code to purposely route it one way or the other (dialaround), and then some other company handles it.

Obviously, circuit switching vs. packet switching makes this different. You don't tend to place phone calls hundreds of different end stations in the matter of a couple of seconds, but when you visit a web site, you're blasting packets to all sorts of destinations. Just think about how many different networks you hit when you load a page laden with "social sharing" buttons, user tracking 1x1 pixels, advertisements, media delivered from a CDN vendor, and so on. That's a bunch!

If you're wondering what gets me thinking about this kind of craziness, I will try to unwind the "stack trace" which got me here.

Earlier this morning, I was awakened by my lovely weather radio alarm, and yes, it was in fact for an AMBER alert. Someone made off with a car in San Jose and there was an 11 month old inside. Bad stuff.

A few minutes after that, my phone started ringing. On the display was the number 999-999-9999. I've seen a lot of phone number craziness, but that's a new one. I hit the "lock" button once to shut it up and figured I'd check it out later. A few seconds after that, I heard the usual "zzt-zzt" of a new text message arriving. It was from some 5 digit "short code" which I had never seen before. How curious!

I did some digging, and it turns out this is all related. I had signed up for an "AlertSCC" service which allows various county agencies to push notices to me. They had apparently picked every single method of distribution, since I got that radio alert, a robo call (which successfully left a voicemail), a text message, and yes, even an e-mail.

It seems the company which pushes these things deliberately sets their calling number to 999-999-9999. This isn't exactly difficult if you have the right sort of telco service, but it is a little weird for a legitimate service to be doing.

This in turn got me thinking about what can be done about so-called "caller-id spoofing". I remembered reading about some company which thought they had come up with their own Secret Sauce which would put an end to such things. From what little I could discover, it seemed they were "plugged in" at a level which gave them full-blown SS7 data for calls and not just a 10 digit number. Then they'd do some magic with this to see if it was legitimate or not.

This was some years ago, and I don't remember who it was. I can't seem to find anything useful about it now, either.

Anyway, to continue the "stack trace", this got me thinking about how calls transit the telephone network, and the kinds of agreements which must exist between companies, and then ... aha ... the agreements which exist between companies and end users. They have to deal with long distance companies directly, or at least, they did. The lines are blurry now, but for a while there was a clear separation.

Then I wondered what that might look like on the Internet, and the rest you can see above.

That's been a hop, skip, and a jump through my brain, cooking up a half-baked idea, and serving it up as a post. Enjoy.

Let's say you're a little company. You have 100 users. You make a change to your product that really annoys 1% of them. Fortunately, 1% of 100 is just one person, so while that one person is a loss, it's probably the sort of thing you can survive.

Your company grows and now you have 100,000 users. You make another change and again annoy 1% of users. This time, 1000 people are miffed as a direct result of your actions (or inactions). That's a bit of a crowd, but you can probably pass it off as "a handful of hard-core users who don't represent the whole". Whatever helps you sleep at night.

Some years down the road, you pretty much dominate whatever sector you are in, and you have at least 100,000,000 users. Again, you annoy 1% of that population. This time, a cool million people are unhappy with you. That's basically the entire city of San Jose, for instance. Or, if you like, that's "1 RI" (Rhode Island).

This might sound familiar, because last year I wrote about a situation where 10% does not equal 10%. In all of these scenarios, you annoy the same percentage of your user population. The problem is that people aren't just some random interchangeable parts. People have connections to other people. Once you have enough people in one group, you have a relatively short number of "hops" to most of the world.

Look at it this way. If you annoy 10,000 people, what are the odds that one of them is going to be a super influencer who can turn others away from your product? If none of them are, how many of them are just one "hop" (like a friendship) away from someone who is? Just how big does that group have to get before it includes someone who is rather well-connected and will roast you over the coals for screwing things up for them?

How about if the annoyance factor doesn't have a linear relationship to the number of users? That is, as the system grows, the percentage of people who will be annoyed by your change actually drops. In that scenario, you should be afraid to use any service that's really getting massive, because it becomes really easy for them to justify whatever they want. They can just say that some tiny percentage of people actually use it, and even though that may be a bunch of individuals, they won't see that or care.

This is why justifications of "it's just n%" don't always sit well with me, even if n is a relatively small number. That could be a handful of people, or it could be massive. If they think that 10 people and 100000 people are the same just because they are both 1% of their respective groups, how can you be sure they are even human? Who thinks of people as strictly numbers or rows in a database? What company has that many users and is also that cold and calculating?

Actually, I think we know the answer to that one.

In the earlier post, I gave the example of having config files be relatively simple and free of assignments, variable references, loops, and most conditionals because I lacked the means to treat them as something "living" like actual code. This time around, I'm talking about the reverse.

Let's say you want to write a tool which will configure a USB device for a user. It's a helper which aims to take care of a multi-step process for them. It's supposed to be relatively portable and flexible so as to support a bunch of different Unixy operating systems and types of hardware.

One of the things you have to do is use magic values which are defined by the operating system or hardware. These typically show up in a bunch of C header files under /usr/include. If you're talking about serial ports, there are magic values like TIOCM_RTS and TIOCM_DTR. When you do things with files, you see values like O_DIRECT and O_NOATIME.

When you're actually writing a program in C, this is no big deal. You just #include whatever header files you need to get the right #define values, and then you use those symbols directly in your program. The preprocessor takes care of the rest at compile-time, and you use the actual magic values which are appropriate for that system, whatever they may be.

Now imagine you're coming at it from the perspective of a rather generic system, and you have a config file where a user says "use O_NOATIME when opening this file". Just think about what you'd have to do to actually handle that sanely. One way would be to have a big lookup table where you essentially map "O_NOATIME" (a string) to O_NOATIME (the #define), and then look for matches when parsing the config file.

The problem is: what happens when the user asks for something which isn't in already in your lookup table? Maybe your system has a feature you want to use through this program, but since the program doesn't have a mapping of "O_SHINYTHING" to the actual O_SHINYTHING symbol, you can't access it.

This is still one of those places where none of the solutions are very good. You probably will wind up writing a glorified "eval" just to extract the value of something. That is, assuming a C/C++ type compiled language, I expect most people will solve it by writing something which implements the following pseudocode:

  // create temp.c for writing
 
  // write to temp.c:
  // (include whatever .h files here)
  // "int main() { printf("%d\n", O_SHINYTHING); return 0; }"
 
  // close temp.c
 
  // compile temp.c to temp
 
  // popen temp, read output, parse to numeric value
 
  // remove temp and temp.c
 
  // associate numeric value with "O_SHINYTHING" internally

Come to think of it, that seems like the kind of thing which happens in autoconf-generated configure scripts. If there's no compiler on the system, I guess you're out of luck.

This probably seems like a bit of a stretch, and it mostly is, but there are still some times when you want a user to be able to switch something on without having a whole bunch of special cases or your own mappings. It's the flip side of the split between data and code.

Earlier today, I started working on building a fairly big module for one of my newer projects. It has client-side Javascript, HTML, and CSS, plus even more happening on the backend. The server has to deal with incoming POST data and cookies, build a request, and then fire that off at a third-party service provider. It has to take the response from them, parse it as JSON, pull out some tasty bits, and then make sense of it. If it all checks out, then it does a commit to its own local database and sends an approval back to that client code. The client then takes it upon itself to rewrite the page with fresh content.

None of this is very special, really. It's the same general pattern you find all over the web to do just about anything you can imagine. There are no mysteries here. The service provider has gone to great lengths to make their API as accessible as possible, with extensive documentation, examples, and just general cluefulness from what I've seen. All of the plumbing is straightforward. It's just a matter of hooking it all up.

I am capable of understanding all of this and performing all of the work. It just happens to be a rather long process with many steps, and each of those steps tends to split up, fractal-style, into many sub-steps. Getting the sequencing right matters here, and thinking about all of the places where it could go wrong and testing for those possibilities is a non-trivial part of the work.

So, when I want to talk about something like this, it's not because I don't know where to start, what I'm doing, how to do it, or even if it's possible. I do know where to start, what I'm doing, how to do it, and yes, it's obviously possible. It's just so much easier to do these things after talking about them and laying it all out in terms of a narrative.

That narrative tends to swirl around and loop back on itself a bunch as those individual pieces are subdivided, laid out, and implemented, but you get the general idea. There's a journey from point A to point B, and while it may sport a whole bunch of important details, it will happen.

When I don't have someone available for discussion purposes for whatever reason, that's when my "lab notebook", marker board, or even diary type things come into play. I tend to keep a running log of what's going on as it happens, including calling out things which have purposely been ignored for the time being. Maybe I know that it's not totally correct to do X, but hard-coding it right now will allow me to proceed on four other things which are far more important.

I only write about this now because I became self-conscious of this running log while working on this earlier and flashed back to a particularly bad manager interaction I documented a couple of years ago. This was a guy who assumed that if you were talking about a problem, you couldn't solve it yourself and needed help. No, dude, I just needed to put it into words to get more of my brain involved with sorting it all out. If I needed assistance, rest assured there would be actual questions in the mix and not just descriptions of what is going on.

(I will add a side note here: talking to actual people is a great way to find out potential "gotchas" in a field you might not know about, and that's always welcome. But, that's not what I'm talking about in this case.)

I'm not going to second-guess myself just because someone doesn't get it. I keep logs of my development work and I'm proud of it. So there.

...

As for my project? It's not ready yet. I took a time-out from it to get this post written for the day, and since I'm flirting with the "stupid hour" here, it will be a while longer before I announce what it is.

...

It's the distant future. You're there, naturally, and you're on this big space station. Think of the majestic music from 2001. Really, it's an intergalactic Denny's, with lots of travelers stopping by for a rest or to grab a bite to eat.

Along with humans like you, there are many species of aliens just doing their thing. There's this one species in particular which is rather prolific, just like us. They have also spread out across their solar system and into the stars and there are billions of them just like there are billions of us. The other species we tend to encounter aren't nearly as numerous from what we can tell.

There's a darker side to this big group of aliens. There's a tiny little subgroup hidden within that likes to experiment on humans. I'm talking about the "anal probe" stuff of lore, where they go and do all sorts of invasive things to figure out what makes someone tick. It's a secret society hidden within this much larger group because it's illegal and they want to stay out of prison.

You can't identify the experimenters just by looking at them. It's not like some of them have green eyes and others have blue eyes, or some have three arms while others have two. They just *are*. The only way you know if any given alien of that species is an experimenter... well... is to find yourself on their exam table. Then you can be pretty sure which one you have.

So in any case, you're on this space station, and you go in to the restaurant part to have a meal. As you come in, you hear some aliens of this large species talking. Maybe they're speaking your language, maybe you know theirs, maybe there are chips implanted in your head, who knows. All that matters is that you understand exactly what they are saying. They aren't exactly using their "indoor voices".

While waiting on your food, you eavesdrop, although it's actually difficult to *not* hear what they're saying. They're discussing something and they're clearly not in agreement. One of them says "they're all the same", while the other goes "there are differences". The first one says "no, you're wrong" and the second one says "no, white ones do this, black ones do this, Asian ones do this, well..."

It becomes obvious they are talking about varieties of humans. You eventually realize their conversation is actually about the sort of experiments they could perform to determine this or that about a subject and answer their "racial questions" about human physiology. It occurs to you that they really got going when you were seated at your table and they noticed you.

Before that point, while you were waiting for a table, they had been talking about something else. You didn't remember what it was, but it's clear that the conversation changed as soon as they spotted you. They latched on to the whole "hey, there's one" angle and ran with it, and didn't care who heard. Your very presence in this intergalactic watering hole has changed their behavior, and now they are using you as a basis for their future experimental ideas.

For the rest of your meal, you are now very aware of their locations and general demeanor, because otherwise you might wake up on a lab table with the shiny metal probes and other horrible things like that.

The thing is, it might turn out these two aren't even experimenters. They're just wannabes of this secret society. They like pretending to be members of that illegal clan because it makes people fear them. You noticed them, and they noticed you noticing them, and they enjoyed it. It was the dessert to top off their wiggly alien meal.

So... hey, wait, this was supposed to be science FICTION.

Shit.

Remember this post?

What happens now?

Why start something if you know there's nothing but badness at the end?

Here's another ramrod story from the world of databases.

For some reason, this guy who was notorious for installing horrible things on customer boxes got the idea that he would start running reports against the company's big database. I forget exactly what got him going on this, but since I had experience with that by virtue of creating my many analysis tools, he came to me with his questions.

He also came to me with news, like when he was proud of having done something. Some of these accomplishments were dubious at best. One day, a chat window from this guy popped up in my face.

X: I just added "day of week" columns to the query for the (table) that match the created_date and closed_date

I responded, saying that I didn't understand, since Postgres will extract that for you. Still, he continued.

X: 2002-05-23 12:34:56 = "Thursday"

Groan, so he was using literal strings? He kept going.

X: In separate columns... didn't know pgsql had a function for that, I'm more an Oracle / MSSQL / MySQL type

He supposedly knows three different flavors of SQL implementations and he didn't go looking for a function? That's, what, 30 seconds with a search engine?

Now I was curious, so I prodded.

R: extract (dow from time) as dow

R: i'm doing something like that

R: did you actually write a doomsday function?

By "doomsday function", I meant something which implemented the Doomsday rule for mapping dates onto days of the week. I've written code like this before out of desperation, and know that any time I even think about doing it now, something is VERY WRONG.

That's why I was asking if he actually came up with his own way to map "2013-03-20" onto "Wednesday". Remember now, this guy is from the "ramrod" category, and so you know any solution is going to be completely crazy.

X: I used a kludge - awk the yyyy-mm-dd to date and have date convert it, seemed quicker than writing it myself

X: external system call from perl

I didn't ask for details, but I think we can probably infer what's happening from this. First, he has data coming back from Postgres. It's a timestamp, since the people who set up the schema for that particular database knew how to use the types properly. That means you get back something like this:

postgres=# select now()::timestamp;
            now             
----------------------------
 2013-03-20 14:55:33.484445
(1 row)

So then, he takes that string and washes it through awk, presumably to strip it down to just "2013-03-20", or maybe to split it up and move it around, making something like the US-style "03/20/2013", or who knows what.

Did you catch what he said earlier about an "external system call from perl"? This guy was already in a language which was built around the notion of crunching text and sliding characters around to suit you, and yet he shells out to awk to do his text mangling?

See, I told you he was nuts.

Then, after he has the YYYY-MM-DD, it's another external call to 'date' to mangle it into a weekday. I assume it was one of those things where you call date with the -d switch and pass it a string, like this:

$ date -d "2013-03-20"
Wed Mar 20 00:00:00 PDT 2013

Then, I imagine he took the abbreviated weekname and mapped it back to the full one, since notice what he said way back in his first example: "Thursday", not the "Thu" that date would give you.

Oh, and finally, he wrote all of this out as a comma-delimited file.

Did I mention that commas were common in our data? Lots of customers had names like "Foo Bar, Inc." or "Widgets, LLC" or whatever. Treating them as a separator was a recipe for extreme pain.

One thing which stands out at me, looking back at this scenario: at no point did this guy go "wow! that's so much better!" and rip out his awk + date crap for a simple "extract...". Oh no. He kept it in there.

postgres=# select extract (dow from now()::timestamp);
 date_part 
-----------
         3
(1 row)

See, it's easy!

If you're that attached to code just because you wrote it, even though it's completely pointless and wrong, you might just be a ramrod.

Yes, they have an option for a "courtesy wipe". You think they'd find some other way to describe it. As far as I can tell, this option is so that it will do another pass or two after you use the option which sprays wiper fluid on your windshield.

In my own car, there's no option for an extra pass. It just happens. Of course, it's usually not enough unless I happen to keep up a certain minimum speed to drive the wiper fluid backwards. Otherwise, gravity takes over and sooner or later I have streaks down the windshield and it's time for yet another pass with the wipers.

Of course, using the wiper spray feature at speed is not advised if you're anywhere near any other cars. It will wind up splattering cars well behind you. That in itself is one problem, but there's more. There are people who will do this on purpose to indicate their displeasure with someone who's following them too closely, for instance. Thus, if someone receives this little spritzing from you, they may think you are doing it deliberately and try to retaliate somehow.

I would also advise against using this feature in the drive-through lane at your local fast food restaurant. I saw it happen once, entirely innocently, and it was rather embarrassing for all involved. There's not some magic force field which keeps that stuff on your glass, after all. It goes all over the place.

"Courtesy wipe" just sounds way too personal.

Regarding my Mac keyboard rant, Isaac wrote in to let me know that the "circle-up-left-arrow" maps to Escape. I guess that sort-of makes sense once you know it, since you can make the case for it "escaping", but that's a pretty big stretch. When I think of escapes, I think of people breaking out of jail, or cats escaping from cages, or that things like that. Not circles and arrows.

Sean pointed out that my CGI example showing an IP address was only both C and C++ compliant if you assumed C99 since I defined a variable in the middle of the block. I forgot all about that. It's funny, because when I purposely set out to write C, I deliberately park all of my variable declarations up top as if C99 never happened. My only excuse is that I wrote it, then eyeballed it and said "hey, I bet this could work as C, too", fed it to gcc, and it worked, so then I wrote it in the post.

This is actually important since some compilers apparently are forever stuck at C89, so you can't go off and use those newfangled C99 things. Yep, this should sound familiar.

An anonymous reader asks if I ever found an alternative to Comcast. I'm sorry to say that I have not. It's not that Comcast is bad, because at the moment, they are doing their job just fine, but rather that I have no alternative. If they start misbehaving technically, politically, ethically, or in any other way which bothers me, there aren't many options.

Maybe I should just start micro-trenching my own fiber from here up towards one of the *many* co-los in town. Then, once I make it to one of those sites, I'll just sit there and look pathetic until someone lets me in and lets me terminate it into their peering system.

That seems like a web site which should exist: a bunch of rogue people who climb telephone poles, scoot through drainage pipes, and use a bunch of rights-of-way for their own purposes. There would be pictures of them acting like they belong somewhere and having close calls with actual utility workers and maintenance crews.

I got a report that Amazon apparently isn't selling my book in certain Asia/Pacific countries. Unfortunately, this seems to be beyond my control. I set it up as "worldwide rights" (since I obviously created all of it) and have a price set for all of their little country-specific sites, but there seems to be some other problem in certain locales.

If that's just an Amazon thing and isn't some kind of local restriction on what you can buy and from who, you might try Barnes and Noble, since as of a few weeks ago, it's on the Nook, too. I'd even figure out the whole Apple thing and push a version to iBooks if it would help someone. Let me know if that someone is you.

Finally, an anonymous reader asked what would happen with the $5/mo vhosting customers when they got hacked and used a bunch of bandwidth. I had to check with one of my friends who also worked on that account back in the day. Neither of us could remember ever having to deal with that problem for that customer. It's probably not so much that it didn't happen, but rather that it didn't matter.

This is because I messed up and forgot the prices they were charging. They were charging *10* dollars a month per domain, not 5. That means those dozen (or so) machines with about 1000 domains each were grossing $120K/month. If they had an overage, it probably wasn't worth their trouble to worry about it. They'd just pay it and feel no pain.

Oh, sure, from time to time they'd have us check on the box and deal with real trouble spots (like things keeping ports 80/443 bound due to mod_php and friends), but otherwise they seemed pretty laid back about the whole thing. If I was making that kind of money per month just by keeping the machines running, I think I would be pretty chill, too.

Five dollars per domain per month might not seem like much, but when you can get 1000 or more domains on a single server and you have a dozen servers, suddenly you're talking about some serious coin. I assume those customers weren't paying us $60K/month for those 12 machines, and so were raking in quite a nice profit for a setup that basically ran itself.

All of this automation came with some costs. You had to give up the notion of being able to just drop in and modify Apache config files to your heart's content. The whole thing lived in a database, and making a change in the control panel really just changed the database. There were then tools which would read the database and would write out Apache configs, qmail configs, and anything else which might be needed underneath.

They had this fairly deep scheme of nested includes to make the web server configs work. First, you had your normal /etc/httpd/conf/httpd.conf, but at the bottom, it would Include /some/path/to/httpd.include. That file was the "top level" Plesk include file, and it was dynamically generated. It would be rewritten any time anything changed with any of the web sites, or basically any time it felt like it.

That top level file would then include individual files for each virtual host, also automatically-generated. Those were also called httpd.include but they lived in the individual home directories which were associated with a given client and domain. The same basic thing applied here: they could and would be updated at any time with no warning.

So, when someone came along and tried to do something weird with their Apache config, life got interesting. A frequent request was when someone wanted to twiddle some aspect of their PHP configuration across their entire web site. Normally you'd just stick some directive in the VirtualHost container in httpd.conf, right? Well, that's no good here. Some people would just edit httpd.include, and it would work for a little while, but then it would eventually be overwritten and would die again.

Sometimes people really bashed their heads against this wall for a while trying to figure it out. I even saw things where someone had used "chattr +i" to make the httpd.include file immutable just so their evil hacks would stay around. Plesk's file creator was able to deal with the inability to write that one file without breaking the whole thing, I guess.

At some point, they introduced this feature where you could create your own little "vhost.conf" and that would then get Included for you. This finally gave people a way to do those magic settings without having to make system-level changes which affected everything in parallel.

I should mention that a lot of these PHP-related config requests were bad news. People would ask for things like "allow_url_fopen", claiming they had done their homework and wouldn't be compromised. Of course, a few days later, their site had been automatically scanned and defaced by yet another worm which then turned around and started launching attacks from there.

I mean, when you can effectively make a script run any command you want on the web server by pointing it at your evil URL, you should expect bad things to happen. You're practically giving them the keys to the kingdom right there.

The other one we'd get a lot back in those days was "open_basedir". I'm not much of a PHP person (phew!) but it seemed to be something which restricted where file operations could happen. Whatever the default setting happened to be wasn't compatible with some scripts people wanted to run on their $5/mo account, and that lead to a lot of manual config mangling by those of us working as web support monkeys.

The last one I remember seeing a fair amount was register_globals. Yes, this is the thing where you can basically let incoming requests define anything they want in your namespace. Some crazy people would ask for this to be enabled, and we'd try to stop them, but a few would persist, and it would go on the box. More often than not, they'd get owned, too.

I guess this could just be selection bias, since people don't tend to open support tickets for stuff which actually works. On the other hand, I sure saw a whole bunch of machines running malicious code over the years. I don't think most of these users had any idea what they were in for when they enabled some of this stuff, and nothing we said would make any difference. They would not live without that guest book.

I have a "simple" task: take a time string and turn it back into a time_t value. The input string is unambiguous:

Tue, 01 Nov 2011 10:54:00 -0700

Likewise, the format string is also unambiguous:

%a, %d %b %Y %H:%M:%S %z

I have three different machines here: a Mac running Mountain Lion (10.8.2), a Linux box running Slackware64 13.37, and an OpenBSD 5.2 (amd64) box. (I actually have a RHEL 5 box handy too, but since it yields the same results as the Slackware machine, it's not being enumerated separately).

Using strptime() with that time input and that format string yields a "struct tm", and it's populated like this:

	obsd	slack	mac
tm_year	111	111	111
tm_mon	10	10	10
tm_mday	1	1	1
tm_hour	10	10	10
tm_min	54	54	54
tm_sec	0	0	0
tm_wday	2	2	2
tm_yday	304	304	304
tm_isdst	0	0	1
tm_gmtoff	0	-25200	-25200
tm_zone	(null)	(null)	PDT

The first part is totally fine, but it goes off the rails when it comes time to figure out the offset from GMT. The "-0700" indicates seven hours west of GMT, or basically, my local time zone in the winter.

The Slackware box and my Macs pick up on this and populate the "tm_gmtoff" field with the offset as a number of seconds. The Mac goes one step further and takes a guess at the time zone using its local name: "PDT".

The problem is that my OpenBSD machine punts on this entirely. It doesn't do anything with the "%z" in there, apparently, even though the return value from strptime() indicated that it processed the entire string.

It gets better. If you take this "struct tm" as supplied by strptime(), you might think you could hand it to mktime() to get a nice time_t value. According to the man page, it takes a pointer to a struct tm (this should worry you, since it's not a const pointer, but just a plain old pointer - more about that later), and it returns a time_t.

Okay, so, let's call mktime() and see what we get.

	obsd	slack	mac
mktime	1320173640	1320173640	1320170040

Before we continue, for reference purposes, here's that input time again:

Tue, 01 Nov 2011 10:54:00 -0700

I'll translate that to UTC because it comes in handy later:

$ date -d "Tue, 01 Nov 2011 10:54:00 -0700" -u
Tue Nov  1 17:54:00 UTC 2011

10:54:00 -0700 (PDT) turns into 17:54:00 +0000 (UTC). This makes sense.

That time, no matter whether it's PDT/-0700 or UTC/+0000, is 1320170040. That's the only way to express it as a time_t.

Now let's look at the results.

OpenBSD and Slackware gave me 1320173640, which is 3600 seconds (one hour) too high, and translates to 18:54:00 UTC. The Mac gets the expected value of 1320170040. A one hour difference says "DST" to me.

The Linux man page for mktime() says that a positive value in tm.tm_isdst means DST is in effect, while zero means it is not. A negative value means it should use the time zone info and system databases to try to figure out whether it's active or not.

OpenBSD says the same thing: positive is in effect, zero is not. Negative values make it try to "divine whether summer time is in effect", and "it may give a different answer when later presented with the same argument". Fun!

There's something interesting in the OpenBSD manual, though:

timelocal() is a deprecated interface that is equivalent to calling mktime() with a negative value for tm_isdst.

Linux mentions timelocal() is nonstandard and should be avoided. However, it also says this:

The timelocal() function is equivalent to the POSIX standard function mktime(3). There is no reason to ever use it.

It curiously neglects the "... with a negative value". In any case, it's nonstandard and deprecated, so I won't be using it.

One thing I can do is make a copy of the "struct tm" and whack tm_isdst down to -1 before handing it to mktime(). I'll let it use its local ruleset to see what happens.

Now the results seem to agree.

	obsd	slack	mac
mktime with tm_isdst hack	1320170040	1320170040	1320170040

There's an interesting bit of code in the Linux timelocal/timegm man page. It says that you can get a portable version of timegm by just clearing the TZ environment variable (!) before calling mktime. Yes, really.

So, okay, I make yet another copy of my clean struct tm and do the "clear-the-TZ" craziness before calling mktime. This is what happens:

	obsd	slack	mac
mktime with TZ hack	1320144840	1320144840	-1

OpenBSD and the Slackware box agree, and give a value which is 25200 seconds ahead -- that's 7 hours. Basically, they interpret it as if it was 10:54:00 UTC instead of 10:54:00 PDT.

At the same time, the Mac punts and fails, and I get a -1 back.

It turns out that if you also whack tm_isdst to -1 at the same time when doing this TZ hack then the Mac plays along and gives a result.

	obsd	slack	mac
mktime with TZ hack and isdst hack	1320144840	1320144840	1320144840

It's still the wrong value, of course, but hey, notice how it's exactly offset by the tm_gmtoff value! You might be tempted to do something like this:

time_t t = mktime(&tm);
time_t actual_time = t - tm.tm_gmtoff;

t (from mktime) is 1320144840, tm_gmtoff is -25200, so 1320144840 - (-25200) is 1320170040, and that's the value we want.

But... beware of this technique. Remember earlier when I said it takes a non-const pointer to your struct tm? mktime can and will modify the struct. One of the changes I've noticed is that sometimes it will zero out the tm_gmtoff field.

If you just rely on that value being there after a call to mktime() you may be surprised. I can't actually trigger this at the moment, but I know I tripped over it earlier while fighting with this problem. If you intend to do something like this later, you have to make a copy of that value separately before the mktime() call.

That's pretty much the situation here. You start down this dusty desert road of trying to figure out something which works everywhere, and before long you're up to your armpits in rattlesnakes.

I'm sure I've missed quite a few things here, and I'm guessing that upon saving and publishing this post, I will return to my code only to trip over yet another problem from another format or another Unix system.

Snakes. Why did it have to be snakes?

Initially, all it did was grovel around in the usual boring log format emitted by Apache (and similar tools). It would add up either hits or bytes and would give you a sum at the end. It was a little like 'wc' in that sense, only it understood the "field" nature of the logs in question, and would handle byte counts appropriately.

In other words, if there was a hit for foo.jpg which is 500000 bytes and another hit for bar.jpg which is 250000 bytes, it would report 2 hits and 750000 bytes of transfer. It had just enough domain knowledge about those often-tricky ASCII logs to be useful.

Of course, it didn't end there. Someone found they needed to whittle down the input data. Normally that would be a job for grep, but since all of this stuff was happening inside my program, I had to add an argument to let you specify expressions. Then you could say "-e foo" and it would only add up all of the hits matching "foo". This came in handy for only a subset of the traffic in a log was interesting.

This made people pretty happy, but later on I got a request to make it handle the "xferlog" format from whatever FTP daemon was common on those boxes at the time. The same basic ideas prevailed, and sooner or later I had something which could also digest that format and emit similar results, along with the same expression-based filtering as before.

I don't think I ever got to the point of having it chew on mail logs, like those emitted by sendmail or (gasp) qmail. Having a message count and a cumulative byte count probably would have been useful in a few situations, but it was well outside the 80-90% of cases which I had knocked out with just web server and FTP logs.

One idea I had but never acted on was writing a helper tool which would grovel through the Apache config files and would follow things like "Include" directives. It would do this traversal to identify all possible log files. Then you could apply a filter to that list to get a subset. That list of logs could then be fed to the first tool in order to get counts from them.

In this sense, you could have a customer on the phone asking about hit counts for www.foo.com, and instead of trying to figure out where he hid it, you could just "greplogs --hits `findlogs www.foo.com`" and let it run.

Obviously, yes, you could do all of this with various combinations of tools like find (or even locate), grep, and awk. Now, try explaining how to build a multi-program pipeline and the awk language to someone who still right-clicks and selects "copy" in their terminal while on a Linux box running X (hint: select and middle-button!). You could do that, or you could just point them at this little binary and let them run queries, confident that it's locating and parsing things correctly.

When there's a never-ending stream of tickets, phone calls, and customers who want results now, with little time to actually train anyone, the all-in-one tool tends to win over the Unix glue approach. I'm not really proud of this, but that's the way things worked out.

Still, I find some of these logs interesting. Look what appears for the first time on Friday morning:

72.14.x.x - - [15/Mar/2013:07:33:56 -0700] "GET /w/atom.xml HTTP/1.1" 200 461363 "-" "rss2email/2.71 +http://www.allthingsrss.com/rss2email/" "rachelbythebay.com" "-"

Yes, that's a Google IP address. Someone there, probably on the corp network, set up rss2email for my feed, perhaps because Reader is going away.

It kept going into the afternoon, then disappeared. I hope they're not trying to run a cron job which accesses their home directory from their workstation. That can end badly.

Still, sometimes there are signs way up high which have been grandfathered. There must be some very specific rules about how much you're allowed to change, since it winds up producing things like this:

Yep, up until a couple of years ago, the topmost sign on this pole was in fact for an Italian restaurant. They apparently went out of business and were replaced by an Indian restaurant... a vegetarian one, at that. I'm guessing they're not allowed to change the outline or height of the sign, but they obviously can change what it says.

And so, you wind up with a huge "boot" of Italy... for an Indian restaurant. I guess that kind of real estate way up high on El Camino Real is worth the potential confusion.

Here's another example from a different part of El Camino. Can you guess what this used to be?

Give up? I'm pretty sure that's an old Pizza Hut sign.

As long as I'm talking about Pizza Hut, here's a weird old one I caught back in 2000 down in Texas. I don't know how this thing managed to stick around so long. The restaurant building itself had the normal (for that time) logo, complete with the droopy double-Z thing which you can see in the outline above.

I suspect that sign is still there. That kind of huge, tall, gaudy thing is exactly what you won't find around here for any new construction and basically any old construction which has replaced their signs.

Without some kind of control on the situation, you end up with this:

It just goes on... and on... and on...

With that in mind, here's another twisted creation of mine.

Think of an online service you use. It could be your web mail provider, your RSS/Atom feed reader, your favorite source of cat pictures, or something else entirely. How many "eggs" do you have in that "basket"? How bad do you think things are?

You don't have to guess any more.

Now you can find out.

Have fun!

When I tried doing this for myself back in 2011, the first thing I tried to handle was Atom feeds. I figured they were just XML, and while I'm no fan of that, I could just hand it off to libxml2 and figure out how to speak its language of "accessor" functions. There must be some way to get the feed-level info, then some kind of iterator into the individual posts, and so on.

The first thing I learned was that the so-called "null namespace" is not your friend. Until you overcome that, libxml2 will act like there's nothing in your document. You can try and try to get "//feed/title", but nothing will happen. So, you manually cram it into the "atom" namespace, at which point all of your references start looking like "//atom:feed/atom:title". I wish I was joking, but I'm dead serious. It took me a while to figure that out.

Anyway, once I got through all of this, then I had something to finally extract usable data from a feed and make it available to other things. There's a lot of stuff you wind up having to learn the hard way.

I guess all of this was on my mind a couple of weeks ago when I got really bored and came up with the completely nutty idea of protofeed, in which I serialize my posts as a binary protocol buffer just because I can, and it's far simpler than wrangling XML. Then I put that binary blob online in case anyone wanted to play around with it.

Then yesterday happened, and the true fate of Google Reader finally made it to the outside world. And so, I decided to do a series of recordings in which I demonstrate writing a really horrible little program which will fetch a 'protofeed' file and then parse it. It even pulls up one of my posts in a browser within the terminal just to show how it works.

If you've been following along with these introductory "lesson" sessions, then please join me for another. I'm also happy to announce the new index page for all of this, which should make finding new entries much easier for all interested parties.

Here it is: rachelbythebay/edu.

This latest batch of recordings also includes a short sequence on how to get protobuf linking working in 'bb' (via .build.conf), so if that's something you want to do with one of your own projects, you definitely should check it out.

This should sound familiar. It was only a matter of time. When internal corporate priorities shift, sooner or later, things are going to get the axe. There are now a whole bunch of users who are about to be permanently left in the lurch and who need to start chasing down alternatives.

Personally, my solution was to just sit down one night and write my own. It took a while to get it fine-tuned to handle the variety of existing feeds, but it was time well spent. According to my changelog, I haven't had to do anything to fred's source code since November. I put in the work once and now it pays off every day.

Do I want to write replacements for every single thing I use? No, not really. However, the alternative of having to hop around every time someone changes things really stinks.

I'd rather operate on my schedule, not theirs.

All I'm doing here is pulling a variable from the process environment to see who might be connected. REMOTE_ADDR is set by Apache when you wake up as a CGI. It's important to notice that this can fail, particularly when running it standalone (from the shell).

I also did a HTTP request to it over IPv6 just because I could. The program doesn't really care either way, but it does provide a warning for the future: REMOTE_ADDR might not always be a dotted quad. You can and will get IPv6 addresses with hex characters and colon separators if someone hits you that way. They might even do it on purpose to see if they can break you.

Recording #1: fetch and display REMOTE_ADDR to visitor if possible.

...

Aside: I wonder how long it will be until someone discovers an exploit where they put some x86 bytecode in their IPv6 address and use it to break into a program which really isn't expecting anything longer than "xxx.xxx.xxx.xxx" in that field. Smashing the stack v6?

It started out like any other Sunday morning. I had just parked myself in front of my workstation to check for mail and say hi to any friends who were online. I started typing out a command and then my monitor went to sleep all by itself. It came back on a second later just in time to see the usual reboot sequence starting. I had no idea what was going on, and just watched it.

The usual BIOS stuff played out and then it went to wake up the SCSI adapter and scan the chain, at which point it rebooted again. Now I knew something was definitely going on. The first time could have been explained by the OS going crazy or someone breaking into the box and commanding it to reboot, but rebooting ... during a reboot? That had to be something else.

As I watched, it did it again and again, and once, it even ejected the CD from my CD-R. This was an older drive which used a caddy, so this involved a deliberate "whirrr-THUNK-CLICK!", followed by the caddy at the front of the case. This made no sense.

I turned the box off and turned it back on, just to see what happened. It did the same thing. It still found all of the SCSI devices, but fell over every time the scan finished. I cracked the lid of the machine to have a look inside. I found something which resembled dust, but it was thicker than usual. It wasn't going away when I blew into the machine. I had to actually poke it with something substantial to make it move.

I figured maybe the machine had inhaled too much dust while sitting on the carpeted floor, and that had caused something to overheat. It was plausible, at least. I decided to clean it up and start swapping parts around to see if I could make the problem follow any one part. First, since the SCSI chain seemed to be so closely correlated with the problem, I pulled out the adapter and swapped it for an older one I had on a shelf.

This seemed okay, and I managed to boot off a floppy which had support for that other adapter. That let me back into the box, and I recompiled my usual kernel to make it support this temporary adapter. That went just fine, but when I went to run lilo, it fell over again. Now I didn't know what to do.

Maybe it was memory. I started up memtest86 and let it go. A bit later, I looked back, and it had detected an error. I only had a single stick of RAM in the thing, so it's not like I could do much about it. I dropped into the BIOS settings for the memory. There were all of these crazy settings like CAS and CL, and I thought I might have had a mismatch with whatever memory I had given this thing. With it set back to "BIOS defaults", it managed to run a pass without failing.

I went away for a while to have dinner, and when I had returned, it had failed yet another pass even with these new settings. Then it failed again. I kept fiddling with settings, slowing things down more and more. I didn't have a separate RAM tester, so what now?

Well, I still had my multimeter. I put it across one of the drive connectors and found I had a nice 5.00 volts on the +5 line and ~11.9 volts on the +12 line, at least until the SCSI adapter came up. Then it would drop to ~11.8 volts and stayed there until it crashed, at which point it would return to ~11.9 volts. Of course, I had never taken readings when the machine was healthy, so I couldn't say whether this was normal or not. It was data but it didn't necessarily mean anything.

After seven hours of fiddling with this ridiculous thing, I finally punted. I had "desktop publishing" type work to do and obviously couldn't rely on this machine. I had a laptop which could be connected to an external monitor and keyboard. It was annoying to use for this sort of work, but it would get the job done. My broken box would have to wait.

Over the next day or two, I tried all kinds of nutty things, including even running all of the parts without a case. I wanted to see if there was some kind of grounding anomaly or something shorting against the case or whatever. It didn't help.

I had one other machine which had compatible but slower memory (PC100 vs. PC133). It was my only source of alternative memory at the time, so I took down the other box and tried it. That didn't help. I think I tried another power supply, too, and that didn't help, either. Now I had a pretty good list of things which had been tried and didn't help.

Swapping the memory didn't help. Removing all of the add-on cards which could be removed and replacing those which couldn't (like the video card) didn't help. The problem never followed any part to another machine. The only things I hadn't been able to change were the CPU and motherboard. It had to be one of them, or maybe both. Nothing else could reasonably be blamed for this.

A new CPU was several hundred dollars while a new motherboard was only about $100, so the solution was obvious. I ordered a similar board which would accept my existing CPU with overnight shipping and got it at my doorstep about 24 hours later. I installed the new board and all was well. It never crashed again, so the CPU was obviously fine. It must have been the motherboard.

...

A few months passed. Then, one day, there was a story on Slashdot about a capacitor epidemic. A bunch of Taiwanese-sourced parts were going bad -- some actually exploded -- and they were taking out motherboards. I went back into my files and took a closer look at the pictures I had taken that morning when the machine first died. I noticed an interesting "dot" at the top of one of the capacitors in that picture:

[Cropped version. Click to see it uncropped and larger.]

That seemed odd but it wasn't a sure thing. I didn't know if capacitors were supposed to look like that or not. Then, one day months after that, I happened to find a picture I had taken inside the machine long before that point, and it happily included that same area of the board.

Now it all made sense. In this picture from a year earlier, there was no "dot" on top of that capacitor. Finally, I could be reasonably sure of what had happened, and why my machine had suddenly checked out that one morning. It didn't change anything, but at least I had some closure.

Computers: what an incredible time sink.

I first heard about the "herd" in the context of having lots of server processes all trying to handle incoming connections in parallel. The situation might go like this. First, you open a socket (stream, not datagram), bind it to a port number, then set it to listen for incoming connections. Then you fork off a bunch of children who all inherit that file descriptor and proceed to wait for incoming connections.

Apparently, when a single connection would come in, all of the children would wake up, but obviously only one would ever succeed in the subsequent accept() call to actually answer the connection. The others would find nothing to do and would loop back and go to sleep. This unnecessary waking and sleeping of processes which had no real work to do added overhead to the systems and slowed them down somewhat.

This particular memory resurfaced recently upon hearing about a failure mode in a relatively large distributed system. It seems that it would have an enormous latency spike every day at 9 AM local time (for the reporting user in Brussels). At that point in the year, 9 AM there was midnight here on the west coast of the US -- DST hadn't kicked in here yet. It's surprisingly common to see tech companies using US/Pacific as their time base, and so having something run at midnight isn't much of a stretch.

It made me wonder if they had some kind of scheduled process like a cron job which did a config push at midnight. Of course, for that to have the kind of impact this user was reporting, there would need to be a synchronous waking of all of the instances coupled with some kind of nontrivial work which blocked out user requests for a noticeable interval. It's not exactly the "thundering herd" from the performance wars of the late '90s, but it's a similar concept.

I started thinking about what you'd have to do to design a system to make it hang up like this. One way would be to have a dispatching service which handles all of your config files. All of your clients connect to it and register callbacks for receiving updates. That is, when your code initializes the library for this service, it also says "run my 'NewConfig' function whenever this changes". Later on, when a fresh config file comes out, the updates fan out to all of those clients, who all receive the blob of data and wake up in their NewConfig() functions.

Usually you don't want configuration details to change while a request is being serviced. This would be a problem with a multi-threaded program, for instance. You get around this by having a mutex which blocks reads from the config file it is being updated. A request handler (which can be run at any time via the threading model) might look like this pseudocode:

HandleFeature(request) {
  ReaderMutexLock l(&config_mutex_);
  if (!config_.feature_enabled) {
    // return error
  }
 
  // actually do something and send reply
}

During normal operations, these handlers wake up, grab a "reader" lock, and that's no big deal, since all of the other handlers are doing the same thing. There can be any number of reader locks open in parallel since they won't conflict.

However, when it's time to apply a change, the NewConfig function has to take a full read/write mutex which will block everyone else:

NewConfig(new_data) {
  temp_config_ = ParseConfig(new_data)
  if (!temp_config_valid) {
    // return error
  }
  MutexLock l(&config_mutex_);  // blocks everyone else
  config_ = temp_config_;
}  // exiting function unlocks mutex

That's the lightweight version, where it does all of the processing (ParseConfig) before taking the lock. It does the absolute minimum amount of work with that lock held: it copies the config in, and then it returns (which, incidentally, runs the MutexLock destructor, which calls config_mutex_.Unlock(), and that unblocks everyone else, in case you were wondering).

Here's another way to handle the same situation:

NewConfig(new_data) {
  MutexLock l(&config_mutex_);  // blocks everyone else
  config_ = ParseConfig(new_data)
  // do stuff with config_
  // run a bunch of update functions
  // generate statistics
  // write many counters to the log file
  // push a status report to disk
}  // exiting function unlocks mutex

This one does the parsing and a bunch of other things with that mutex held, so as long as it's "thinking", anything else which might need to obtain a lock on that same mutex is blocked. This means your request handlers might be unresponsive.

Now, this in and of itself does not have to be the end of the world. A process which takes a break when it gets a new configuration file doesn't have to bring down your entire service. You can have other instances of that same process running, and your load balancers can merely steer the requests to those which are still available.

The problem comes when you mix both this "lots of work with a mutex held" and the "thundering herd" in the same system. Now you have a situation where all of them wake up simultaneously, receive the update, take their respective mutexes, and start applying the update in parallel. While this is going on, your poor load balancers can't send the requests anywhere, since all of your replicas are out to lunch at the same time!

This is the kind of stuff which leads to user-visible delays.

What can you do about it? That all depends on what your system is supposed to do. If you have a situation where you really, truly, need all of your instances to run in lock-step in terms of configuration, you don't have much choice in the matter, now do you? You just have to suck it up and have all of them update in parallel. You can try to minimize the pain by making the updates run as quickly as possible, but there will still be some period of time in which your availability is technically zero.

It would be better if your system could handle a mixed configuration, even if only for a relatively short interval, but that sort of thing typically has to be included early in the design of a project. When you have that kind of flexibility, you can just update your instances gradually and let them run in small batches (or even one at a time). Even if your load balancer isn't smart enough to notice this and redirect hanging requests to other instances, it'll still only affect some small percentage of users. The others will be hitting replicas which have already been updated or haven't yet been updated and won't have any lag. The ones who do see lag will only see it briefly, since it's likely their next request will be served by another replica, or in the case of "sticky sessions", their server will finish and will go back to normal.

If you tell all of your receptionists to go to lunch at the same time, don't be surprised when nobody answers the phone for a while.

As it was explained to me, an "extender" is really just another name for a calling card. Of course, here in 2013, that probably doesn't help, since who actually uses or has heard of a calling card?

Here's the general idea: you'd talk to your phone company (which probably was the one and only Ma Bell back then), and they'd hook you up with a 'calling card'. This had a number on it which probably resembled your home phone number but might have had a difference or two. Then it typically had another three or four digits at the end for "authentication".

It turned something like "408 555 1212" into "428 555 1212 9324". You'd use this to make calls from wherever you were (say, a pay phone) to anywhere you wanted, and it would be billed to your home phone. That way, you didn't have to try to pump quarters into a cranky pay phone or stick your host with an outrageous phone bill. Keep in mind that long distance charges were non-trivial at the time, and people tended to keep stopwatches by the phone so they wouldn't ramble on.

It should become obvious why these were prized by the people who did all of the long distance dialing to move around new releases of pirated software. All of those calls to Europe weren't exactly cheap, so why not stick someone else with the charge? The problem was one of finding the actual number to use.

You could do it by hand, in theory. First, you'd pick a long distance carrier to target. If you were going to scan for cards issued by MCI (remember them?), the access number was 950-1022 -- that's a Feature Group B number. You'd ring that up and it would answer with some kind of tone, and then you'd key in the number you wanted to call, with a 0 first instead of a 1, like this: 0 415 555 1212. It would make some other tone, and then you'd key in the "428 555 1212 9324" calling card number and wait a moment or two.

If it worked, it would give another noise (I think they were two short blasts of a dial tone), give you a ka-chunk or click, and then the call would go through. You'd get the usual ringing, or busy, or whatever, just like if you had dialed it direct as 1 415 ... and so on.

How do I know this, you ask? Easy. No, I didn't go scanning for these things. That would have been illegal. I had legitimate access to one at the time, and I used it enough to where the whole sequence is burned into my head.

There's an obvious problem with scanning manually, and that's taking forever to try all of those combinations. Even if you could twig on to a phone number which probably has a calling card account, then figure out how it got mangled (408 to 428, or whatever), you'd still have to brute-force the 10,000 combinations the four-digit access code gives you.

This is a job for a computer. There's just one question: how do you have a modem figure out when the call went through? These things were relatively stupid and couldn't identify much in the way of error noises on the phone line. If you were lucky, your modem might detect "RINGING", but that wasn't well-supported.

One thing you'd usually be able to count on was detection of a busy signal. Even modems which didn't have the high end features of being able to detect "VOICE" or "RINGING" would usually have a "BUSY" result. That reduced the problem nicely. You'd only get a busy signal if the call went through, and that means the code worked! Any other result meant you got some error message and it didn't work.

That leaves only one question: how do you make sure you're always going to get a busy signal, so that your scanner can detect a valid code? There were test numbers which were always busy, and some people knew about those, but there was another trick, too: just call your own number. Since you're obviously already on it, it's always going to be busy.

With all of those things figured out, it was just a matter of starting it up and letting it run. Any busy signals would be logged and could be checked out manually to ensure they worked properly. Then these stolen numbers would be traded around in order to procure favors or access to "elite" boards where the actual "couriers" hung out.

Thinking about it now, I'm surprised this went on as long as it did. Anyone who reviewed the audit logs for a given access node at the long distance company would have found hundreds or thousands of call attempts to the same number... all local, and always busy. What kind of nutcase uses a calling card to call a number in the same city? Then they'd look more closely and notice they were being charged to entirely different account numbers. They'd also find even more failed attempts to dial the same number which didn't pass the authentication stage. If they happened to trace it back, it would all be coming from one line, but odds are, they already knew the number, because they were dialing themselves.

Talk about a giant neon sign that says "come here and get me".

All you had to do was start up the program on your computer and wait.

Here's part of a typical Mac dropdown menu, taken from the Mail application which comes with the OS. I call your attention to the list of hotkeys (or "accelerators", perhaps) down the right side of that menu.

First up is this cloverleaf looking thing and a Z. That's not too complicated since there's actually one of those symbols on my "command" keys. It obviously means command-Z, and trying it gives the expected results.

Next you have this up-arrow, then the cloverleaf, and then the Z. There's no up-arrow on my keyboard, but I'm sufficiently experienced (ahem) to have used an actual clickity-click typewriter back in the day, and so linking it with "shift" isn't too much of a stretch. Back then, depending on what sort of machine you had, when you hit shift, things actually, well, shifted right there in front of you. So, okay, shift-command-Z.

The next three are just variants on the first, and then it gets weird again. Cloverleaf plus ... left arrow with an X in it. Again this calls back to the typewriter, sorta. You could back up but you sure couldn't delete what was there without some manual intervention. I know that character appeared on the backspace key of some typewriter keyboard I encountered somewhere along the way, but it sure isn't on my Mac. This is command-backspace, or I guess, command-delete, as the key is marked on my machine.

Then there's just another simple one, and then it gets even stranger. Now it's this ... sleeping cat face thing and what appears to be the power button logo, only rotated a bit and with an arrow on top. That first one is really weird, but I think I found out by trial and error over the years that it means "option". There's absolutely nothing on my option keys suggesting this, but empirical data suggests that's what it's for. The other one? I still don't know. I know it's not the power button, for sure. Nothing else on the machine even remotely resembles it.

Everything else is just a variant of something already covered. There are also menus which use a caret (^) to mean "CTRL", but being indoctrinated in Unix means I know that one already. I can't imagine what it's like for someone who has no idea what that thing means.

What's really annoying is that I'm pretty sure older Macs had more symbols on their keys. I can't remember what they all were or where they lived, but I'm willing to bet they matched up with all of these mystery meat icons in these menus. Why they're using icons for things that don't match anything you can actually type, I may never know.

...

One last thought here: Apple II keyboards inspired a certain sense of exploration. Every kid was told "control-open-apple-reset" to reset the machine and get it booting your disk. I obviously noticed there was a "closed apple" on the other side. We didn't use it much, if at all.

Still, it got me thinking: if the "open apple" works with control and reset, then what does the "closed apple" do? I don't think it did anything, but then that made me wonder what would happen if I used both. Control - open apple - closed apple - reset. I got a crazy system test: the screen went nuts and the little squeaker in the case started making this horrible noise. The computer teacher probably thought I had killed the poor machine.

I knew enough to know that a mere four-fingered "death grip" shouldn't kill the box, and give it a proper reset sequence to shut it up and put things back to normal.

(I didn't know about the PET killer poke back then.)

As a result I don't usually think about things like UTF-8 support. I know I have my web pages set up to deliver things like that, but I also know that nothing I publish really exploits it. I never worried about making sure I had a compatible terminal or anything like that.

Things changed yesterday when I got some feedback about one of my programming recording sessions which asked about UTF-8 support and C++. I had always treated such things as a black box, and never really thought about what would happen if I actually tried to use a multibyte sequence. This took me down an interesting road.

I had long known that 'xterm -u8' would give me a terminal which could render UTF-8 sequences in a meaningful way. I only used it when I had something which looked obviously wrong on my ordinary (non-UTF-8 compatible) rxvt just to see what the character was supposed to be. xterm itself has a couple of UI warts which don't work for me, so I couldn't use it for everyday life.

I managed to find "rxvt-unicode" yesterday, aka urxvt, and it maintains the rxvt behavior I'm used to while adding proper support for UTF-8. It also otherwise looks exactly the same, so I didn't have to adjust to yet another weird user interface regime. Using it, I was able to put together a couple of demo recordings to show a totally basic use of multibyte characters in a C++ program.

Recording #1: I print three bytes in a formatted string: 0xe2, 0x82, 0xac.

Recording #2: I actually directly input a Euro symbol (by pasting from another session, since my Linux box's keyboard doesn't have an obvious way to make one of them).

Recording #3: I use the \u20ac escape sequence.

This only shows that you can in fact emit the sequence and have it look correct if your terminal is set up for it. These little CGI programs should also look fine in a browser since they send the right sort of Content-Type header. The playback stuff also groks UTF-8, so when you view these recordings, it should render properly there, too.

What it definitely does not do is interpret the data in any meaningful way. It's just a meaningless blob. If I tried to use traditional string manipulation techniques on it, particularly the "character at a time" stuff, I would be in for an interesting surprise.

For now, I'm just going to leave them as opaque blobs. We can come back to actually using these multibyte sequences properly another time.

I predict some other people will get goofy about the date on April 31, 2015.

Further, I expect to see the usual posts about MM/DD/YY vs. DD/MM/YY.

All I can do stand back and point at ISO 8601. It's hard to beat.

I figured it was just a TV show and didn't think much of it. However, a comment regarding my in-browser terminal session stuff got me thinking about it again. It had no cursor, and watching it, you can definitely sense something is missing. So, in order to set things straight, I dug into the term.js code and figured out how to get it working.

There is now a nice little underline cursor which scoots around the virtual terminal, following my every move. You can see where I decided to cursor up to a line and then backed out of it and went back to another location. All of it is faithfully reproduced right in your browser.

I also fulfilled some other feature requests from this same anonymous individual. In addition to the existing "back up" and "forward" buttons which moved you 10 positions in the playback stream, there are now single step buttons for both directions. If you really, really want to see every little keystroke and nuance, now you can.

Finally, I added some more feedback on how far you are in a session. The counter now appears next to a total so you can tell where you are out of the entire stream. Right below it, there's now a time counter showing how much real time had elapsed from the beginning of the session where you are now, and next to it, the total time for the entire session.

I hope this makes for a nicer experience for everyone.

Well, around here, during certain times of the year, it's not that simple. If you're the wrong kind of bird, look out!

I've created three bite-sized recordings to show how to do a relatively simple task with an Apache web server: emit something dynamic via CGI. If you've never touched this sort of technology before, my hope is this will remove some of the mystery surrounding it.

Recording #1: the no-frills CGI program with just a tiny bit of dynamic content.

In this session, I created about the simplest shell script I could think of which still creates variable output on the fly. It just says something and then prints the date and time.

Granted, a "real" web program would do more than that, but the theme here today is "small steps". I intend to show how you can ramp up from these introductory stages to creating anything your heart desires. Everyone has to start somewhere.

Recording #2: the same basic CGI, only this time it's in C++.

I wrote a little C++ program which does just enough output to show the general concept. It's so simple, it doesn't even do the time... yet. It has a placeholder for that.

Recording #3: adding date and time output.

This one returns to the code from recording #2 and makes it actually read from the system clock and build a formatted time string. This string is then used in the output instead of the placeholder. It's supposed to resemble the output from the original script I wrote in #1, above.

I use my build tool to make the C++ stuff happen without having to worry about writing Makefiles. If you'd like to see how to install and use that, just check out this earlier recording in which I download a copy and use it to compile a short test program.

Comments? Questions? I'm all ears.

Have you considered becoming a tech writer in San Francisco? You're a good writer. And since there's never more than one San Francisco company with a tech writer at any given time, you don't lose time fretting over whether you should switch.

I don't think I'd go for it as literally defined (taking a tech writer job in SF) but there are some concepts buried in there which seem interesting. I like writing, obviously. I like writing about technical stuff, and I like helping people. I also like getting "inside" a system to figure out how it works... or, in some cases, why it works. Or, if it's broken, I figure out why. There are countless examples in my posts from the past two years.

There's something special about looking at a system you have never seen before. With "beginner's mind", you get to see things other people might miss because they have been using it all along, through all of its various stages, and they are used to how it behaves.

So, I am making a limited time offer to forward-thinking individuals or concerns in the Bay Area. I'll dig into one of your systems and build a technical document and/or report on it. If you like it, then we can figure out payment details and/or other projects for me to analyze.

Here's a real example from my life. A couple of years ago, I transferred jobs inside the bigger company and wound up on a team which did automated testing of the Linux kernel. My new manager asked me to install a new instance of this software on one of my machines to get familiar with it. While doing this, I documented all of the gunk I had to go through to get it working.

Later on, when it was time to set up more of these things, all of the weird little nuances, hurdles, pitfalls, and other anomalies were right there in my documentation. Anything I had encountered was listed, along with whatever I did to either resolve it or work around it. Some of these things wound up spawning bug reports or feature requests. Sometimes, we determined that some behavior was no longer necessary and certain dependencies could be removed to improve the install process. It took a fresh install and an outsider's perspective to catch that.

Happily, I can do that same thing for arbitrary projects. One of my friends pointed me at an iOS application a couple of months back and wanted my take on it from a UI/UX perspective. I fetched a copy from the app store and went to it. I caught a whole bunch of things that he probably saw at some point but had "lost" as time went by.

Essentially, I can behave like one of those users who takes everything literally even though I know better. Then I can turn around and report my findings along with recommendations on what to do about it based on my knowledge and experiences.

Give me two weeks and I'll give you a completely new perspective on your own systems. Just imagine the possibilities.

Linus's Law says that "Given enough eyeballs, all bugs are shallow".

My eyes are ready to step up and look at some new things.

Are you in?

A couple of years ago, I found someone who was talking about solving some kind of puzzle. I kept a small piece of it in my notes as an example of just how "out there" some solutions can be. It looked something like this (any transcription errors are mine):

def turn_to_left
  case @directions
    when 'N' :  @directions = 'W'
    when 'E' :  @directions = 'N'
    when 'S' :  @directions = 'E'
    when 'W' :  @directions = 'S'
  end
end

There was some context, too: apparently you're supposed to be simulating some kind of space vehicle which roams the fourth planet from the sun. (I'm being deliberately vague here, and am not using the term "m--- r-v-r" because I don't want this post to be found for those searches). You are given the dimensions of a flat space, your X and Y coordinates, your initial heading, and a series of commands.

The commands are simple enough: turn left, turn right, or move forward. Turns are just simple pivots, just like the good old turtle used to do, and they're always 90 degrees. You only ever move in the four cardinal directions, in other words.

Anyway, given the initial conditions and the list of commands, you're supposed to say where you end up and which way you're pointing at the end.

Now that you know the context, look back at that snippet of code and feel the pain. Apparently he started worrying about "branching" and was asked to do the job without that case-when block and some other if-else block too, and wasn't able to give them what they wanted.

This is about the point where I started thinking I was pretty weird since I did not think of this problem like this at all. The directions aren't these four discrete things. They are just part of the spectrum which is a circle around your position.

They give you commands as letters. Big deal. A L is a -90 and a R is a +90. See where this is going yet? Do a little modulo magic on them to keep them in the appropriate range. Maybe add 270 instead of subtracting 90, then always take it mod 360. Whatever. The point is, it's a heading, so think of it as just one number, not a series of letters, or enums, or ... well, I've seen a lot of crazy things. I went looking for other implementations, and I think one person had a different class for each direction the thing could be pointed. No, I'm not exaggerating.

Again, they give you a starting direction as a letter: N/E/S/W. Those translate into 0, 90, 180, 270. That's nothing special, either.

There's only one part left, and that's the movement command (M). For that you have to take your heading (one of those four numeric values) and somehow turn it into a meaningful adjustment to your position. These changes... deltas, if you will, are simple enough. Your X position will only ever change by -1, +1, or not at all (0). The same applies for your Y position: -1, +1 or 0.

So now there are these numbers swirling around. 0, 90, 180, 270. -1, 0, 1. This is when "sin" and "cos" pop into my head. They'll take those headings and will give you very nice values which can be used to adjust your position.

The only mildly funky part with this so far is that sin() and cos() in C think in radians, not degrees, so you have to remember 10th grade math and figure out how to convert again. (I hated that stuff, so I had to look it up when I got to this point.) Degrees * PI / 180.

Anyway, fiddling around with sin and cos will show that one of them gives the expected values for X changes and the other does the same for Y, so you do "x += (the one)" and "y += (the other)", more or less. There's one more step involving the fact that you probably won't get exactly -1 or 1 out and will need to do some rounding, but that's part of the fun of floats!

That's the whole "engine". Give it an initial X and Y, heading, and command sequence, and it'll give you the final X, Y, and heading.

Still, if you go looking for this online, you will find countless examples of implementations which go absolutely gaga with their "enterprise grade code", "design patterns" and more.

Reading those comments just blew my mind. It's like, I know all of the words you're using, but those sentences make no sense to me. One solution had "six main components". I found out that by "components", they meant "classes". They somehow found a way to turn that dumb little rotate and move thing I described into a six-class program. Another one had a "HeadingFactory" and even an "UnknownHeadingFactory", and again, I assure you, I am not making this up.

Most of my interest in looking at these other solutions online was to see if anyone had used a heading and trig functions. I never was able to find anything of the sort. Everything I do find is along the lines of "case NORTH: y += 1; break;".

Here's how you can find dozens of posts about this, most with code attached. Just use your favorite web search engine to look for "LMLMLM" and then append "LMM". Again, I'm purposely not putting the whole 9-character string in here verbatim since I'd rather not have this post come up on that topic.

I can think of one final bit of insanity for this whole situation. I bet there are companies out there who actually want to see the implementations which are piled higher and deeper with "enterprise" design goo. If you show them anything shorter, they think something's wrong with it.

I mean, a program which is short can't possibly work, right?

Now bring up that same intersection in a convenient mapping program or web site. The direction they're talking about here is northbound Park to eastbound Otis. Notice what happens with the road there.

Way back before Walgreens, the road goes from being two through lanes to having a dedicated right turn lane and the remaining lane which then later splits off a left turn pocket. Normally, a dedicated right turn lane would imply special treatment at the light. You might put one there to let right-turning vehicles "slip around" the ones who are queued up to go straight. This dedicated right turn lane is actually longer than the left turn pocket!

If right on red is no longer legal, then there is absolutely no reason to let cars slip around. Dump that dedicated right turn lane and realign things so that the left turn lane lines up with the #1 (left) lane back by Walgreens, and that the remaining lane is marked with "straight or right" arrows.

They don't even have to commit to this all the way at first. Just get a bunch of construction barrels, fill them with water or sand or whatever, put some flashing lights on top, and plop them down between the "straight through" lane and the curb. Basically, force people to get bottled up behind traffic which isn't turning. This should introduce some friction to the situation and stop the actual "bad behavior". If it doesn't pan out, they can remove the barrels trivially.

Just remember what I've said about too many users being wrong. It all comes down to what they're trying to do: stop turns on red, presumably for ped safety, or write tickets?

Last night, I got kind of bored, and yet had a really weird idea at the same time. I had been reading a post on HN about some service providing a XML feed of user data because they were shutting down. Apparently they had done a few things which made it difficult to actually use. Then, in the comments, I spotted something which made me feel a little self-conscious:

CDATA sections ... usually hint that the coder doesn't know what they're doing.

I immediately flashed on my own atom.xml feed for this site. I basically made it by cargo-culting some other feeds, and yes, it has CDATA in it. I think I looked at Daring Fireball and a few others. I didn't want to deal with actual XML generators back in 2011 when I wrote all of this, so *gasp* I just emit it "raw". There. My secret is out.

Anyway, this got me thinking about better ways to do things, and then that's what ultimately gave me my insane little idea. What do I use when I don't want to deal with XML in my own life? Easy: Protocol Buffers! They'll handle all kinds of stuff, have a nice compact binary representation, require no escaping of funky characters, and can still be expressed as ASCII for debugging purposes.

While in this loopy state, I came up with the following "minimum viable protobuf definition":

package feedspec;
 
message Feed {
  optional string title = 1;
  optional string link = 2;
  optional string unique_id = 3;
  optional uint32 last_updated = 4;
  optional string author = 5;
 
  message Post {
    optional string title = 1;
    optional string link = 2;
    optional string unique_id = 3;
    optional uint32 last_updated = 4;
    optional string content = 5;
  }
 
  repeated Post post = 16;
}

Then I was really bored, so I sat down and wrote something to actually populate one of these things with the data from all of my posts. It wasn't too difficult, considering it's the same basic idea as my Atom generator. All of the data is already there, and all of my little helper libs make the rest rather easy.

About an hour later, I had something which brought my crazy idea to life. Expressed in ASCII, the beginning of the feed looks like this:

title: "Writing"
link: "http://rachelbythebay.com/w/"
unique_id: "rachelbythebay.com,writing-2011"
last_updated: 1362515359
author: "rachelbythebay"
post {
  title: "\"Everqenote\" Corporation and the Filet Mingon"
  link: "http://rachelbythebay.com/w/2013/03/05/everqenote/"
  unique_id: "rachelbythebay.com,2013-03-05:everqenote"
  last_updated: 1362479183
  content: "<p> Call me picky, but <a href=\"/w [...]

Is it perfect? No. Will it express everything that Atom or RSS could? Nope. Not even close. (But it could be expanded. That is the nature of protobuf, after all.) Is it really, really simple to create and read back in? YEP.

If you're also feeling bored and would like to try fetching and using the output from this scheme, you're in luck. I have the binary serialized output from this process online for anyone who wants to give it a spin. It's still subject to my fine-tuning, naturally. It's only 12 hours old.

I will point out one important part of this whole thing. I deliberately used "string" instead of "bytes" to create an invisible hand which forces you to use UTF-8 everywhere. If you try to feed it a string which is not valid UTF-8, it will barf... and that's the whole point.

Have fun!

"Everqenote Corporation". Really? WTF is that?

As for the mail, I got it because I installed it once some time back and they just had a security breach involving user information.

I guess this means some dumb not-used-anywhere-else password was compromised. Boo hoo. I never used it because I didn't like it, and it was only installed for a short time. Now I should see what it takes to delete that account.

I wonder if all of these mails have this same craziness. Comments welcome!

...

Bonus pic: this one is from a couple of years ago. I turned in their "steak feedback" form, but I gave them a little extra feedback, too: