More stories about stacks of modems
Back in January, you might have seen my post about a very large BBS which seemed to solve for the problem of "how do we host a bunch of users" by having one physical box per user. It might have been one physical box per *two* users, maybe, but it's hard to say.
Anyway, I have some pictures like this from my own life. They show a lot of tangled-up wires and a bunch of cruft like that, but in general, we didn't have one box per user. We had one decently-sized box for everybody... at first.
Now, keep in mind digital cameras weren't easy to come by, so I have very few actual pictures. I'll have to do a lot of explaining to make up for the lack of detail and alternate angles. Clearly, I had no notion of ever using these pictures to tell stories 25+ years later.
It's the summer of 1995. I was asked to build a BSD/OS box that would be "the box" for this brand new network a local school district was building out. The people in charge wanted me to set it up like the box that had been at my high school for a couple of years. It ended up being a Pentium 90 with a Digiboard for eight external serial ports. We bought 4 consumer-grade Sportster v.34 modems to drive those dialups. Later, we bought four more for a grand total of eight analog dialin ports.
Some friends at a local company had some bandwidth to spare (a T1 to Alternet somewhere, as I recall), so they parked a Zoom modem on one of their boxes and plugged it into a phone line. We got an identical model to avoid cross-brand issues (Zoom modems were awful that way...) and used it to dial them. That was our Internet pipe for the first year or so: v.34 (~28800 bps) into them, into their T1.
This is more or less what it looked like:
In this picture, you can see the "glorious" ASA Pentium 90 with both kinds of floppy drives and a CD-ROM (for no particular reason) and a DAT (DDS) tape drive for running backups. On the wall, you can see where *9* different analog phone lines were punched down into jacks: one for each dialin port, and one for the dial *out* to our "ISP".
There are also a pair of Ethernet ports and cables present: one orange, one white. One went to the machine shown here, and another went to some desktop machine not shown. It's hard to tell, but they *are* labeled. Go us.
Sitting on its side is the cursed Zoom modem being used as our uplink.
You can't really see them here, but on top of the case there is a black "Digibox" that fans out all of the serial ports and then eight of those Sportster modems, each with a flat satin cord running back to the wall, and a fairly thick serial cable running back to that Digibox.
There are also a pair of power strips hidden back there with OH so many of those stupid AC adapter "wall wart" bricks: one per modem.
(You'll see some of this insanity in a later picture.)
This box right here did *everything* for a while: dialups, shell accounts, SLIP and PPP access, DNS, mail (incoming and outgoing), it ran the first few web pages, it monitored the rest of the network, and so on and so forth. It just sat there and ran and ran and ran. Every week we'd feed it a tape, and this turned out to be a good thing, since one of the drives died on us and we had to restore from that tape.
That was life back then.
Let's jump forward to 1997. We bought a system based around a 200 MHz Pentium Pro to replace the original Pentium 90 from 1995. The plan went like this: I get BSD/OS 3.0 running on the new machine (a HP Vectra Xu, for anyone who cares), get all of the users and whatnot ready to go, and then swing them over in about 5-10 minutes of downtime.
This took some doing involving port forwarding, NFS mounts and all sorts of other demonry I'd rather not get into... but it worked. Most users never noticed this. What they did notice is what happened a few hours later.
After moving the Digiboard (and thus, all of the modems) from the old machine to the new machine, I started stress-testing it. I had some test boxes I could get to dial in and nail stuff up. Part of the point of this upgrade was to get PAP going so our users could dial in and do PPP without those damned chat scripts - you know, wait for "ogin", send the username, wait for "assword", chuckle like a 12 year old, then send the password, that kind of thing. I wanted to see it in action.
So, I got some boxes to dial in, and started flood pinging them and generally pushing lots of bits at them. The box hung. I rebooted it and tried again. It hung again.
The day went on like this, and then the days that followed. We tried all kinds of things. We worked with the OS vendor (BSDI). We went this way and that with kernel patches and config changes. Nothing helped. Finally, I moved the damn Digiboard back into the old box and just upgraded it to 3.0 so we'd have PAP dialups. It was perfectly fine, so it wasn't BSD/OS 3.0. It was something about that HP Vectra. We never did figure that out.
As a result, the old box didn't get to retire in 1997. It stayed online, and since it had all of those modems hooked to it, it couldn't move from that spot, either. We just put the old and the new machines side by side and left 'em there.
Along the way, we also picked up a dedicated machine to use as both a web server and a web proxy since we found out the hard way that some of our IP space could not be routed over large chunks of the Internet. That, too, was another HP Vectra running BSD/OS, and so it ended up stuffed in there with the other two.
The situation looked like this:
Now you can see some of the context that you couldn't see before. Note the stack of modems there: some on the Digibox, some on the new Vectra. If you look closely, you can see the BSD/OS release box wedged between Vectra #1 and #2. There's also a massive battery backup there so we wouldn't die when the site power went out.
At this point, the 1995 server was running dialups, so it was handling SLIP and PPP logins. It also did this goofy thing I rigged up where if you dialed in interactively (like with a terminal program), it would flip you to the new machine to do your shell stuff on there. Technically people *had* shells on the old box too, but I didn't want them using it. It would have been too confusing.
Maintaining these accounts in parallel was annoying, so people tended to not get accounts on the dialin side of things unless they needed one and asked for it. No, we weren't doing YP/NIS.
The 1997 server was doing most of what the original box had been doing, minus the dialups and the web site. You can see it's also running X here with whatever version of Netscape would have been current.
We had fortunately gotten our own T1 to the outside world, so the Zoom dialout is long gone, for anyone wondering what happened to that thing.
There was a fair amount of heat coming out of this corner, as you might imagine.
Let's jump forward four more years to 2001. The modem companies had come out with this stuff that benefited from the fact that most phone lines were digital most of the way until the last few thousand feet to the actual customer. They used this to create *almost* 56K connections in the form of their own K56Flex and X2 proprietary stuff, and then the eventual V.90 standard. The trick was that to support this as a dialup host, you had to have digital lines into your pool. You couldn't do analog on both ends and still have this work.
We wanted that to let our users do (near) 56K from home. We also wanted more than eight dialups and didn't want to add yet another Digibox, still another stack of those stupid modems, and the associated ports that would have to be strung into the wall and punched down into those jacks. So, we pulled the trigger and bought a dialup rack.
This monster box fit in a 19" rack and took these vertical blades. You'd put in modem cards which would then accept a PRI or T1 in the back. This would give you 23 or 24 lines per card depending on what sort of service you fed it. Then you also had a "server" card that basically did the equivalent of what my old 1995 machine did: it authenticated people (via RADIUS this time) and ran PPP for them, and forwarded packets between those lines and the local network. Finally there was this monitoring card for keeping track of the entire setup.
This monster was installed in 1999 but I didn't get a picture of it until 2001. Here it is at that point:
It's the big black box taking up the bottom half of the frame here. The two boards at left are the actual modems (HiPer DSPs, for those USR nerds out there), and the two towards the right are the others described earlier (the ARC and NMC, for anyone who remembers and cares). The last one there at far right is a power supply.
(Also, looking at this picture now, I have no idea why the Bay router up top only had two of its four screws installed. Sketchy.)
This thing took days to get setup because the telco people were incompetent. Yes, I'm just going to say it. They had no idea what the hell we wanted to do, despite us telling them "we want PRI and are okay with 46 dialups, because we don't want robbed-bit signalling, so we get that slight bump in line rate" for months leading up to that point. But, once they finally figured out how to provision their circuits, it was in, it worked, and that was that.
This meant the original machine could finally retire. I don't remember what happened to all of the modems. I think maybe we doled them out to the school board members so they could dial in from home, or something. They had pretty rough lives up to that point, so it wasn't a great deal even for a "free" modem.
The Digiboard and Digibox probably would have been useful for some projects involving lots of little serial connections, but I have no idea what happened to it, either.
Anyway, this is a very limited view of some of the crap I used to run. I never ran a multi-line BBS, but I did wrangle some of the same stuff for other reasons.
Knowing what I know now, would I have done things differently? You bet. Live-swapping the mail/etc server in the middle of the day because I was an hourly employee and so doing it at night was not a good choice? What utter crap that was. Having only the one digiboard and so not being able to test it in the new box without taking down the old one? Also utter crap. Buying Sportsters instead of Couriers for the original 8 analog lines? Using HP Vectras for ANYTHING? Not having a redundant power supply in the Total Control box? Not fully securing that Bay router in the rack?
You get the idea... so don't send me hatemail about how much of a shitty operation it was. I fully admit it was a joke.