Best practices with ssh root user and backing up ssh keys

lowendmeow · February 2021

Hey, I had a question about this. I have a lot of random idling VPSes and some servers and for a lot of them I just have the root login because I'm the only one that accesses the server. On some of those I will SSH in by key (turning off password login) and then log into another local user on there and sudo, and some of them not. In this scenario where it's just me does it really matter if I disable sshing in as root?

I also had a question about private SSH keys. I have passwords on my SSH keys but I don't have a good sense of how to back these up securely and it's a hassle to regenerate them when I get locked out. What are the best practices for storing private SSH keys? I could put some on a physical USB key maybe with an encrypted volume on top of that? Or is just a good password on them enough? For sure I don't want to store any of these in any kind of cloud storage but physical access like a random usb key lying around or that I have on me makes me a little nervous too.

dosai · February 2021

A really thorough ssh section in here,

https://github.com/imthenachoman/How-To-Secure-A-Linux-Server

foxone · February 2021

Change ssh port
Deny password login
Deny root login
Have your user be something not common (e.g. not "admin")

Just print your key on a piece of paper and store somewhere safe, and have it without context so nobody knows it's an ssh key :P

lowendmeow · February 2021

@foxone said:
1. Change ssh port
2. Deny password login
3. Deny root login
4. Have your user be something not common (e.g. not "admin")

Just print your key on a piece of paper and store somewhere safe, and have it without context so nobody knows it's an ssh key :P

That's going to be tough to restore if it's on a piece of paper! That reminds me of the lavabit guy printing out their key in teeny tiny letters...plus another question I had was what's the best practice with SSH keys, should I have different ones for different servers and such? I do for some of them but some of them I just use the same one. But that would make it even harder with printing out..

bikegremlin · February 2021

KeePass comes to mind:
https://keepass.info/
If anyone thinks it's a bad idea, please correct me.

mobile · February 2021

ed25519

foxone · February 2021

@lowendmeow said:

@foxone said:
1. Change ssh port
2. Deny password login
3. Deny root login
4. Have your user be something not common (e.g. not "admin")

Just print your key on a piece of paper and store somewhere safe, and have it without context so nobody knows it's an ssh key :P

That's going to be tough to restore if it's on a piece of paper! That reminds me of the lavabit guy printing out their key in teeny tiny letters...plus another question I had was what's the best practice with SSH keys, should I have different ones for different servers and such? I do for some of them but some of them I just use the same one. But that would make it even harder with printing out..

Why? Print it with some proper font, and it'll be easy to restore. If you wish, you can use an alternative encoding like check58 to prevent mistypes

chimichurri · February 2021

I don't think changing your port to anything else than 22 makes any sense since tools like Shodan scan 0.0.0.0/0 on all ports all the time anyway.

In general I'd say ssh key auth is probably better than password auth. (harder to brute force, if not impossible)

I'm not sure whether setting "PermitRootLogin" to "no" is of any use, as long as you disable password auth. (don't really matter whether you're root or not...)

I would not be completely against backing up your private ssh keys in the cloud as long as you encrypt them again before storing them inside GDrive or the likes. (Although, if you are that paranoid, then you probably do need to have a few physical copies of each key stored in various places in case of a natural disaster/theft/etc.)

yoursunny · February 2021

My SSH private key is stored on all my computers and phones so they backup each other.
There's also a copy in Dropbox. Not encrypted, because I trust Dropbox.

I've been using the same RSA private key since 2011.
I am thinking about regenerating a new key in ECDSA, but haven't got to it.

cochon · February 2021

@lowendmeow said:
In this scenario where it's just me does it really matter if I disable sshing in as root?

Matter? it reduces the risk, helps for a good nights sleep. It's so simple to configure it should be a reflex action.

ChallengeResponseAuthentication no
PasswordAuthentication          no
PermitRootLogin                 no
AllowGroups                     mygroup

Restart ssh and test BEFORE logging off.

I have passwords on my SSH keys ... it's a hassle to regenerate them when I get locked out.

SSH keys plural? for most scenarios one key should be enough, use the same public key everywhere, security of the local private key is all that matters. If you get locked out of a system, generating another new key won't help much.

but physical access like a random usb key lying around or that I have on me makes me a little nervous too.

If you're nervous about carrying the private key around on a USB stick I can recommend buying a YubiKey or similar and using the GPG module to store the authentication key for SSH. Older YubiKey 4's go for ~$20-30 on e-bay, current 5 model is quite expensive for no advantage [here], many models don't support GPG so do check first.

If you've never used GPG for SSH key authentication (with or without a YubiKey), it's a bit of a learning curve, but worth the effort. You can then login from any borrowed system without long term risk, the secret key material can never be read/copied out of the device once loaded, though it's wise to keep an offline backup securely at home. The device itself can be PIN protected against theft, locks up on multiple (3+3) attempts.

An added bonus on top is you can use libpam-ssh-agent-auth to authenticate sudo actions using the same secure device instead of typing in your password (keylogger?) or risking passwordless sudo. It's a whole new level of peace of mind.

Plenty of tutorials online, e.g.
https://github.com/drduh/YubiKey-Guide
Pay attention to the bits about backing up the key FIRST, the transfer process wipes the working copy, and it's a write only process.

cochon · February 2021

@yoursunny said:
There's also a copy in Dropbox. Not encrypted, because I trust Dropbox.

Do you mean 'not encrypted' as in no additional encryption, or 'not encrypted' as in not even a password on the key? A strong password is still reasonably secure for most threat models, other than keyloggers, but no password at all would be a gift to the prosecution if you were worried about the legal consequences of a data breach.

Trusting Dropbox themselves isn't really the issue, a breach is more likely from some zero day exploit or malware on one of your own devices hooked up to Dropbox.

yoursunny · February 2021

@cochon said:

@yoursunny said:
There's also a copy in Dropbox. Not encrypted, because I trust Dropbox.

Do you mean 'not encrypted' as in no additional encryption, or 'not encrypted' as in not even a password on the key? A strong password is still reasonably secure for most threat models, other than keyloggers, but no password at all would be a gift to the prosecution if you were worried about the legal consequences of a data breach.

Trusting Dropbox themselves isn't really the issue, a breach is more likely from some zero day exploit or malware on one of your own devices hooked up to Dropbox.

I don't have password on SSH private key. I'm not concerned about data breach because I don't handle PII on the server, and the website itself is open source.
I have a laptop and a phone linked to Dropbox, and they both already have the SSH private key.

OsirisBlack · February 2021

I keep my SSH Key with this fella. I'm pretty confident it's secure.

lowendmeow · February 2021

@OsirisBlack said:
I keep my SSH Key with this fella. I'm pretty confident it's secure.

what about this little guy?

lowendmeow · February 2021

@yoursunny said:
My SSH private key is stored on all my computers and phones so they backup each other.
There's also a copy in Dropbox. Not encrypted, because I trust Dropbox.

I've been using the same RSA private key since 2011.
I am thinking about regenerating a new key in ECDSA, but haven't got to it.

so do you use this for all your servers? trying to figure out if i should have separate private keys

@bikegremlin said:
KeePass comes to mind:
https://keepass.info/
If anyone thinks it's a bad idea, please correct me.

I already have a system for passwords that works pretty well - check out Lesspass or Supergenpass. I like this because then I don't have to worry about losing the password database. It's meant for URLS but I have used it for hostnames on devices to manage the ssh passwords

lowendmeow · February 2021

@chimichurri said:
I don't think changing your port to anything else than 22 makes any sense since tools like Shodan scan 0.0.0.0/0 on all ports all the time anyway.

Yeah this is why I'm asking the question people have lots of preferences that they developed over the years, I know some people do port knocking and stuff too but in the end I think some things have changed to make those methods less useful. I generally don't change my SSH port

Wolveix · February 2021

@lowendmeow said: so do you use this for all your servers? trying to figure out if i should have separate private keys

There's no need to have a separate key for each server. I personally keep a key per organization. For example, I have one key for all of my personal servers, one for my employer's servers, and another for my own company's servers.

alwyzon · February 2021

@lowendmeow said:
In this scenario where it's just me does it really matter if I disable sshing in as root?

Not to use root directly is a good protection against your own stupidity. Everybody has a bad day, and you will be happy that you have to confirm critical actions. So yes, I would always recommend to use a super user and try to avoid root whenever possible. Maybe some interesting additional arguments: Server Fault - What's wrong with always being root?

@lowendmeow said:
What are the best practices for storing private SSH keys?

Either use an algorithm that creates short keys (without loosing safety against brute-force attacks) such as ed25519 and print it, or just use an USB flash drive. The next steps depend on the kind of usage of these keys.

Personal SSH keys? Just put the key into your drawer and expect nobody to ever look for it anyhow. Probably put some tag onto it, so that you will actually find the right one if ever needed. Do you maybe have drawer or safe with important documents anyhow? Maybe put it there, that should make sure that you find it when needed.

SSH keys used for production servers in a small business setup? Then put the print out (or the flash drive) into an envelope and sign it (over the seal). Finally, put it into your safe. The safe is there to prevent unauthorised access to the backup, the envelope acts as a log who last accessed the key in case multiple employees have access to the safe (you can't get the content out of the envelope without breaking it). Nothing is more annoying than lost keys because some employee left or lost/broke their primary device and nobody has access to the right keys anymore, so better make it company policy.

@lowendmeow said:
(...) private SSH keys (...)

As mentioned by others above already, there is no benefit of having multiple different SSH keys. SSH keys by themselves are designed to consist of a private and a public part anyhow. The receiving end - which has only the public key - does never see the private key in a public-private-key-auth model. You can think of it as one time passwords based on the private key, verified with the public key. If you don't get scared easily when you see linear algebra, google how RSA works. It's not that hard to understand, and it should be immediately getting clear why you don't leak any information when reusing public keys in multiple places.

Best practices with ssh root user and backing up ssh keys

Comments