Writing

Feed Software, technology, sysadmin war stories, and more.

Wednesday, July 10, 2013

More corporate IT avoidance by geeky employees

This is a story about what happens when you have a whole bunch of techs who do all sorts of web hosting work and an IT department which is more of a liability than an asset. It's about the things which are done that fly in the face of security, reliability, documentation and everything else that's good and proper, all in the name of getting something done.

So there were these techs, and they did mostly web hosting technical support. They took phone calls and they worked tickets. The company had giant pipes to the outside world and a strong DNS infrastructure... as long as you were a customer. If you were an employee, it was like pulling teeth to get hosting space or a hostname in the official corporate domain/namespace. It got to where people just stopped asking, and would tell newcomers to not even bother lest they get branded a troublemaker.

One tech went too far with it and installed an access point to get wireless access on the support floor. They walked someone up there and terminated him on the spot. Eventually, the president of the company heard about it and rescinded it, but it was a sign of the hair trigger they tended to have. So, when things needed to be done, they tended to happen on the "down low" and only the right folks heard about it. That lead to the whole domain name thing.

Every tech had their own workstation (or two or three) at their desks, each with an arbitrary name that didn't really matter. There was no standard operating system so it was a big bunch of weird. Some of these techs would develop useful things: web pages with helpful docs, CGI tools and random multi-user web-based amusements (like the much-enjoyed "paint on the annoying customer's face by clicking" page).

Other techs took on the roles of unofficial "buildmasters", and they started rolling RPMs and other packages for frequently-encountered situations. Customers used to ask for custom builds of PHP, MySQL, Apache, and the usual bits of the "LAMP stack", and it got tiring to keep re-doing it. Having both binary RPMs for the quick load and go situations and source RPMs for those requiring customization eliminated a lot of work.

It would have been helpful to have some kind of way to give all of these hosts names in one or more of the official corporate domain names, but that did not exist. Finally, one day, one of these techs registered a real domain name with an ordinary registrar. Then they set up the domain just like it was any other customer domain in terms of the primary nameservers, and added it to a "customer" account. That is, it was the personal account of this employee, and since customers can host domains, that's how it was added.

Then the word went out: if you want to give your workstation a hostname, now you could. This would let you get back to it from other places inside the company without trying to remember the IP address. You could share the hostname with other people and let them get at your stuff - docs, games, toys, or your massive MP3 cache. Whatever. All you had to do was access this one employee's account, drop into the DNS editor, and edit this one domain.

It took off like gangbusters. All sorts of people added things to it. They even started getting clever and started adding entries for other stuff besides their own workstations. Some of the corporate servers had really stupid and unhelpful hostnames for services which didn't run over HTTP. So, they would set up a memorable hostname in this new domain pointing at the same IP address, and would then just refer to it from then on.

As far as I know, their IT department never found out.

Anyway, this is what happens when you have a bunch of geeks without a useful corporate infrastructure for workstation names or good ways to create memorable URLs. They'll find a way to make one. It might not have anything to do with the IT department, but they'll find a way!