Writing

Feed Software, technology, sysadmin war stories, and more.

Wednesday, October 19, 2011

GT Power BBS netmail suite bag details

I used to run a BBS program called GT Power. It was unusual in that it had its own proprietary network software. It resembled Fidonet in some regards, but other parts of it were very different. Rather than having this stuff slip into the past unnoticed, I've decided to write about it. There's a lot to talk about, but right now I'll just cover the actual files it used to get things done.

The GT netmail suite worked on the basis of files called "bags". Each bag was either an ARC or ZIP file, although it did not have that extension. The entire file name, including the three letter extension (this is DOS, remember), was used as part of the bag identifier. A bag might be B0001015.060 or Q1234045.E00. Understanding the filenames is part of understanding the bigger system.

Network addresses were in the form of nnn/nnn, where the first number was the "net number" and the second number was the "node number". That means an address might be 060/015 or 300/003 or 001/001. Typically, a bunch of machines in a single city would all share the same net number, at least initially. Politics and other wrangling sometimes lead to the existence of multiple nets with overlapping regions.

A similar address scheme used in this network was for the identifiers used for "echomail" or "echoes". Like Fidonet, this term applied to the discussion forums where multiple systems could participate. In the GT network, an echo ID was in the form Enn/nnn. The first E was always there, and the rest was numeric. It might look like E00/045, E10/001, E20/002, E99/001, and so on.

So now that you know what a "net/node" ID and an "echo" ID look like, you can see how they turn into bag names. A "B" bag (we'll get into the types later) of generation 0 with sequence number 001 heading to 060/015 would be named B0001015.060. There were counters kept on the system for various targets, and the bag numbers would be incremented to avoid collisions while in flight.

Likewise, an "E" bag containing a day of posts for the E00/045 echo might be E2040045.E00. You might have noticed the "2040" in that E bag filename. In this case, it's actually a Julian day, where January 1, 1987 is day 0. That corresponds with the approximate birth of the GT Netmail software.

What were the bag types? First, there were "B" bags, which were typically just data going from one node to another. Actual "netmail" messages sent from one person to another would be in there. The message inside was just a plain text file with the same naming scheme as the B bag, but it had an M name, like M00001015.060.

B bags would also be used for echomail messages going "upstream". This is because GT echoes did not work like Fidonet echoes. In Fidonet, messages flood out in all directions from the originating BBS. They use things like "seen-by" and "path" lines to avoid loops.

GT doesn't do this. A GT echo has a "sponsor" and everyone else is just a subscriber. All messages entered on subscriber systems flow up to the sponsor in B bags just like ordinary netmail messages. They might look like B0001045.E00. There, they land on the sponsor's system and sit in the usual message base. Some time later, during a "bagging" run, the sponsor will batch up all messages for a day and send them back out.

This means the sponsor actually had the ability to edit messages or delete them entirely. This didn't seem to be used very often, fortunately. It should be noted that this had the tendency to add latency to discussions since those mails usually needed to wait up to 24 hours to catch the next bagging run after arriving at the sponsor's system.

So an E bag would just be a ARC or ZIP file with a bunch of M files for the individual messages. However, that wasn't the only way you could subscribe to an echo. After a certain point, they added support for Q bags. These were actually tokenized streams and used a shared "wordlist" dictionary file. They were supposedly 20-50% smaller than E bags compressed with ARC because "these archivers do not handle small files like messages well", according to the Netmail docs.

That's one optimization, but there were more. A system might typically connect to a half-dozen others, but there were dozens or hundreds of nodes in the network at some point. This meant that any one connection might have to convey a bunch of bags. The netmail protocol wasn't the fastest thing in the world, and it took on the order of 2-5 seconds to start up a file transfer.

If you're spending long distance money to move these things and it's just sitting there burning time doing nothing, you're bound to find a way around it. The solution was the "G" bag. When you turned these on, it would take a bunch of bags (files) bound for another system and would make a bigger, single G bag out of them. This meant you had multiple ARCs and ZIPs inside of another ARC or ZIP file.

There were a few more types. "A" bags contained a file which was being attached to a message. Someone who wrote a netmail message could put ".FA stuff.zip" at the beginning of a line. When it came time to bag up that message, the Netmail software would look in a specific directory for "stuff.zip". If it found it, it would roll it into a A bag. That A bag and the M file for the message itself would wind up in a B bag as described above.

Sometimes, mail bags would get stuck. Maybe they were routed to a system which then did not have a route to the destination. After some number of days, it would be renamed to an "X" bag. Then it would be returned to the originating system via file attach ("FA"), which means an A bag... which goes into a B bag... and might go into a G bag!

It seems that store-and-forward systems are something of a lost art. I imagine a bunch of the lessons learned from things like GT, Fidonet and Usenet will need to be re-discovered if people start making peer to peer social networks and things of that nature.

Finally, to give some idea of how old this stuff is, here are the release numbers and dates for the GT Netmail suite from the documentation I was able to find:

There is also a version 4.21 (which I managed to decrypt and open in order to make this list), but it doesn't seem to have a changelog or release date available.

Towards the end of the GT network's useful days, a bunch of crazy political things happened and that lead to a bunch of ridiculous technical battles. I'll get into that another time.