Writing

Software, technology, sysadmin war stories, and more. Feed
Wednesday, March 10, 2021

Mac upgrade opened sshd to brute force password attacks

A couple of weeks ago, I read a post about how the "sealed system" on Big Sur was hurting people. I kind of skimmed through it and figured it was mostly complaining about the size of the download. For whatever reason, that hadn't been a problem for me and my machines, so I kind of wrote it off.

Last night, I applied the latest security patches to arrive at Big Sur version 11.2.3, and realized that I should have paid more attention to that thing. It explained something that I had been noticing for a while: my Apache config would keep reverting.

For a good number of years, I've been (foolishly, as it turns out) using the included Apache httpd to run small localhost-only stuff on my Mac laptop. In recent times, I found that every upgrade of the OS would revert my httpd.conf and it would need to be put back.

This was the status quo for a while, and I was considering writing a post on the topic along the lines of "you might as well give up on using that included httpd, since anything you do to it will die regularly". That was my thought on the topic until just a few minutes ago when I realized something else probably happened: it probably reverted my sshd config, too.

Why would it matter if the sshd config got reverted? Simple: it's because the stock Mac sshd install includes password-based auth, and that means someone can brute-force their way onto your machine if they can connect to it on port 22 for long enough.

The last time this came up, I wrote a dumb little checker script, and kept running it against my entire fleet of machines until they all checked clean. So, I ran it again just minutes ago, and son of a bitch, it's wide open yet again.

$ ssh-check host.name
host.name: *** accepts a password for ssh logins ***

The resulting RAAR I let out got me to start this post. This, too, was something I wanted to write about eventually, but this forced the issue.

In the interest of helping people out, here is my terrible little script. It requires bash in /bin, netcat as 'nc' in your path, ssh (obviously), and timeout from GNU coreutils. In other words, it's intended to run on Linux and I haven't bothered to fine-tune it for other places yet, including a Mac.

If you are in fact on a GNU-ish setup, then here, you can just run this awful thing. Beware, you may not be able to unsee this disaster. You have been warned...

#!/bin/bash
 
IP=${1}
 
nc -w 1 -z ${IP} 22 &>/dev/null
 
NC_CODE=${?}
 
if [ ${NC_CODE} -eq 1 ]
then
  #echo "${IP}: not listening on port 22"
  exit 0
fi
 
timeout 5 ssh \
  -o PreferredAuthentications=password,keyboard-interactive \
  -o StrictHostKeyChecking=no ${IP} &>/dev/null
 
SSH_CODE=${?}
 
if [ ${SSH_CODE} -eq 124 ]
then
  echo "${IP}: *** accepts a password for ssh logins ***"
  exit 0
fi
 
echo "${IP}: running sshd but is okay (no password methods)"

The way it works is suitably nasty, too: it assumes that ssh will block asking for a password, so it treats a "hang" (timeout expiring) as a password prompt, meaning it accepts one of those two methods. Now, ssh will also block for long time if it tries to connect to a host that's down or otherwise is dropping packets from you, so the script does an awful "will it accept a connection to me" check first with netcat.

See, I told you it was terrible.

If, however, you can't run this script, you can do your own checking by hand - just type this in and add in the hostname or IP address on the end. TYPE IT IN. Don't copy and paste. Leave off the last part.

ssh -o PreferredAuthentications=password,keyboard-interactive -o StrictHostKeyChecking=no -ZZZZ_I_SAID_DONT_PASTE_THIS_IN

Don't just copy and paste this into your shell. TYPE IT IN... and leave off the -ZZZZ part. I put that there to catch people who paste straight into their shells from random web pages.

I could have run some terrible JavaScript on this page that hijacked your copy mechanism and added something nasty. I mean, I probably didn't, but why would you trust me, or anyone else who has access to this page, server, or the Internet connections between you and it?

[Sidebar: seriously, go to one of these "news" sites, go to copy a stretch from the page, and notice how your cursor goes a little wonky. That's them injecting something into the paste. When you go to paste it somewhere else, it will contain something you *did not* see on the page. Imagine if you ran that as a command. That's what I'm trying to teach you about here.]

Anyway, if you run that command manually and it asks for a password, guess what? You, too, can now be brute-forced.

Now what? How about locking it down? That seems like a good idea.

You have non-password-based logins to your sshd, right? .ssh/authorized_keys and all that stuff? If not, go do that first, or you will lock yourself out of the machine if you follow the rest of this post.

Another thing: if you are doing this remotely (!), for the love of all that is good and holy, have another root shell open and just chilling out in the corner in case you screw up the config, and don't close it until you have verified you can open a NEW connection and get all the way back to a root shell!

...

Assuming you have a non-password-based way into your machine at this point, then continue to turning off password-based auth.

First up, you have to edit /etc/ssh/sshd_config, find "# PasswordAuthentication yes", uncomment it, and flip it to no instead. Save the file. Due to a quirk of how the Mac starts sshd, it'll recognize it for the next connection - none of that "sshd reload" stuff here. (Really. Don't try to poke launchctl or something. Learn from my mistakes.)

Run the check again and you will find that... it's still accepting password authentication. This is because there's a second path, and it happily falls back to that. You now have to look for "UsePAM yes" and flip *that* to "no" as well.

Save it out, run the check again, and you should find that it's fixed.

Congratulations, you have now stopped your Mac from being exploited by the army of ssh-scanning idiots on the Internet.

And, just remember, the next time your Mac does so much as a *point update* for security reasons, you get to do this all over again.

...

Notes on the script: you might've noticed I commented out where it says that a host is not listening on port 22. This is so you can run it against a whole whack of machines at once and not hear about the ones which aren't up.

Here's a really sad way of running it as a batch:

for i in $(seq 1 254); do ( ssh-check x.x.x.${i} & ); done

... you know, that kind of thing. Try it on your own network and despair at the probable state of your ssh daemon configs.

I'm sure there are better ways to scan for this. Feel free to use them and then you can bury my script in an appropriate place. I suggest the bottom of a cat box.