SSH Key Management Tools and Best Practices

Author: Ganesh Velrajan

Last Updated: Fri, Mar 25, 2022

When we were a small team we used to manage SSH keys manually. We used to generate keys using OpenSSH tools like ssh-keygen and copy-paste the public keys to the servers either manually or using the ssh-copy-id tool. As our team grew in size and the number of servers that needed to be managed increased, configuring, managing and keeping track of SSH keys manually became complicated.

Most likely, you are in the same situation now and you are searching for an SSH key management solution to address these problems. As the head of your Ops team, you are interested in an SSH key manager that automates and handles all the complexity associated with SSH key management at scale.

Let me tell you this. You are actually trying to solve the wrong problem. You are not trying to address the root cause of the problem. Want to know more, read further.

What is SSH Key Management

SSH key management involves two things:

  1. Creating, distributing and securing the keys
  2. Key rotation.

Best Practices

Here are some of the best practices for SSH key management:

  1. SSH private keys shouldn’t be shared with anyone.
  2. User private keys should be encrypted using a passphrase.
  3. Avoid using the same SSH keys to login on several host machines.
  4. Copying SSH keys to ~/.ssh/authorized-keys file in the servers should be done using secure copy tools like ssh-copy-id
  5. SSH public keys should be deleted from all servers when an employee leaves an organization
  6. SSH public keys of the servers must be placed in the ~/.ssh/known-hosts file in each user’s machine to avoid seeing the Trust On First Use (TOFL) message.
  7. SSH keys must be rotated periodically, atleast once in a year or whenever a security compromise is detected.

However, it is very hard to follow these best practices. SSH key based authentication looks great in theory but in practice, it complicates IT operations and is prone for errors (we’ll discuss more about it in a minute). Probably, this is why your ops team is complaining about managing SSH keys at scale and you are in search of an SSH key management solution that handles the operational complexity.

Ideally, you shouldn’t be trying to solve the SSH key management problem. The problem is not with managing SSH keys at scale, but with the SSH key based authentication itself.

To understand this statement better, we need to take a holistic look at how SSH key based authentication works in practice.

How SSH Key Authentication Works

SSH key authentication involves generating a public-private key pair using a tool such as ssh-keygen from the OpenSSH toolkit.

The private key should be kept safely and secretly in the user or host’s machine. It shouldn’t be shared with anyone or copied to any other location.

The public key, derived from the private key, can be safely shared with others without compromising the security.

For a user to access a server, the SSH public key of the user must be placed in the ~/.ssh/authorised_keys file in the user’s home directory. Similarly, for the user to trust the host machine or the server, the public key of the server must be placed in the ~/.ssh/known-hosts file in the user’s machine.

This is because, when a server needs to authenticate a user, it will crosscheck the public key offered by the user with the one stored in its system. If it is the same, it will permit the user to login. If not, the connection will be rejected. The same check is performed when a user tries to trust the public key offered by the host he or she’s trying to log in.

Operational Challenges

Whenever you hire a new employee, you need to create and place the public key of the user into all the servers the user needs access to. Similarly, when an employee leaves an organization you need to delete the key from all the servers.

When a new server is brought into service, you need to place the public key of the server in the ~/.ssh/known-hosts file of all users’ machines. Similarly, when a server is taken out of service, you need to delete the key from all the users’ machines.

When you perform key rotations periodically, you need to repeat the above steps on all the user and host machines.

When the list of servers and users increases, your team is overwhelmed by these operational tasks. Their ability to keep everything sane and secure is significantly challenged. This is probably the reason why you are looking for an SSH key management solution.

Moreover, SSH keys have perpertual lifetime and never expire. If some unwanted user were to get hold of your keys, they could continue to use it to login to your servers for years, unless you rotate the keys more frequently.

Ideally, you shouldn’t be using SSH key based authentication. For all practical purposes, you should use SSH certificate based authentication instead.

What is an SSH certificate

An SSH certificate is nothing but an SSH public key signed by the private key of a trusted Certificate Authority(CA) within an organization. The CA is the central source of trust within the organization. The CA transfers its trust in the form of signed certificates issued to users and hosts within the organization.

Moreover, SSH certificates have a validity period stamped in them that makes them expire automatically.

How SSH Certificate Authentication Works

SSH certificates are issued to all servers and users in an organization. They exchange their certificates with each other when one has to trust or authenticate the other.

SSH certificate based authentication requires minimal configuration and setup, meaning, the host or user machines need not maintain a list of all public keys (or certificates) of the users or hosts respectively.

SocketXP Bastion Host Solution with SSH Certificate Management

For SSH certificate based authentication to work, all the servers and user machines must be configured to trust the CA of the organization. This is done by copying just the CA certificate to all the user and host machines.

Thereafter, a server would trust any user certificate issued by the CA and permit login. Similarly, a user machine would trust any host certificate issued by the CA, before logging in to that server.

Open Source Based SSH Certificate Management Solutions:

OpenSSH server and client supports SSH certificate based authentication. You can create CA certificates, host and user certificates using the ssh-keygen tool. However, OpenSSH server doesn’t automate the creation, signing, distribution and rotation of SSH certificates. Basically, it doesn’t offer a turn-key SSH certificate management solution. You need to build your own tool around it.

But if you are looking for a turn-key SSH certificate management solution, there is BastionXP.

What is BastionXP

BastionXP, which is an open source and open standards based PKI/CA, is a turn-key SSH certificate management solution that creates, signs and distributes SSH certificates automatically. BastionXP is built using Google’s Open Source Crypto Libraries.

BastionXP PKI/CA issues certificates based on user or server identity using Identity and Access Management (IAM) solutions such as Microsoft Azure Active Directory, AWS IAM, Okta, OneLogin or others. SSH certificates are issued after an user authenticates successfully via OAuth or SAML based SSO and Two-Factor Authentication(2FA) providers such as MS Office 365, Google Suite, Okta, GitHub, OneLogin etc.

Moreover, BastionXP issues short-lived SSH certificates that expire on a daily basis, eliminating the need to purge the certificate of a user from all the servers when he/she leaves the organization.

BastionXP automates certificate rotation, when a security compromise requires certificate rotation.

BastionXP also supports Role Based Access Control (RBAC) using SSH certificates. This restricts who can access what resources within an organization and at what privilege levels.

I highly recommend you give a try with BastionXP SSH Certificate Management solution and get a first hand impression about how it automates and simplifies end user workflow.

BastionXP trial version is absolutely free to download. Refer to the Getting Started guide here.

Start Your Free Trial Now

Try BastionXP for free with no commitments. No credit card required.