Tightening SSH access using short-lived SSH certificates

Author: Ganesh Velrajan

Last Updated: Tue, Feb 1, 2022

SSH access using public private key based authentication has several drawbacks that could potentially compromise your organization’s SSH access security. SSH certificate based authentication addresses most of these security problems while simplifying certificate distribution. BastionXP Bastion Host solution automates and simplifies the SSH certificate management while offering additional security services such as issuing short-lived SSH certificates, SSH session recording, monitoring, logging and tracking functionalities for auditing purposes.

Traditionally, enterprises use SSH public private key based authentication for SSH login to servers - both in-house and in the cloud.

Usually, IT admins create, copy and set up the SSH public private keys for each host (host key) when the VM or server is brought into service. Similarly, IT admins create, copy and set up the SSH public private keys for each user (user key) who needs to gain access to the VM or server. Sometimes, they even use a homegrown or third party tool to create and distribute SSH public private keys.

Problem:

Setting up a SSH public private key requires so much time and effort. The admins need to meticulously create, copy and set up a public private key for each user (and also for each host) without leaving any relic anywhere.

For example, the user public key needs to be appended to the host machine’s ~/.ssh/authorized-keys file and the user private key needs to be copied to a folder (~/.ssh/) in the user’s device such as laptop or desktop. Note that the user public key needs to be copied to each and every host that the user needs access to.

As a result, the server and user onboarding process is usually laborious, slow and cumbersome. At scale, the problem only gets magnified and the key distribution become really messy.

Here are some security issues associated with using SSH public private key based authentication for SSH login:

  1. Copy of an SSH key pair left undeleted in emails or local folders or shared online storages, while copying to the host or the user’s machine.

  2. SSH keys don’t have an expiry date or validity period stamped in them. So they can be used perpetually. The technology doesn’t force the admins or users to rotate the keys periodically. As a result, there is no real urge to renew SSH keys periodically. Secondly, rekeying and distributing the keys to servers and user machines at scale requires significant planning and execution. This builds up the inertia against rotating the SSH keys periodically.

  3. Some users use the same SSH key pair to login to multiple host machines, potentially increasing the attack surface for any unwanted user who gains access to such an SSH key.

  4. When users connect to a server for the first time, weird cryptic messages are shown to users to identify the host key fingerprint and accept it, so that the host key is stored permanently in the ‘known_hosts’ file (~/.ssh/known_hosts) for future references. Mostly, users blindly accept the message and continue with the login anyway, without bothering to know the authenticity of the host they are trying to log in. The only way to address this security problem is to pre-populate the “known_hosts” file with the host public keys of all known hosts that the user would potentially login to. In large organizations, this list is huge and changes over time, which means the ‘known_hosts" file on each user’s machine should be kept updated as well. As a result, most IT teams don’t follow this approach.

  5. SSH keys don’t have the hostname or username stamped in them. As a result, any host that is setup with the SSH key could pretend to be the actual host.

  6. When users leave the organization, SSH keys assigned to them are usually not deleted from the “authorized_keys” file in all the host machines.

Solution:

The solution to all the above security problems is to use SSH certificates for SSH login and authentication.

How SSH Certificate based authentication works:

Each organization sets up a Public Key Infrastructure (PKI) to issue SSH certificates to its hosts and users. Certificate Authority(CA) is the prime component of a PKI. For SSH certificate based authentication, two CAs need to be created - a Host Certification Authority(Host CA) and a User Certification Authority (User CA). Host certificates are signed by the Host CA and the user certificates are signed by the User CA.

A host certificate and the corresponding private key needs to be stored in the host machine only. Similarly, a user certificate and the corresponding private key needs to be stored in the user machine only. User certificates need not be copied to the host machines and host certificates need not be copied to user machines, unlike the SSH public private key based authentication. Advantages of SSH certificate based authentication over SSH public private key based authentication:

It is sufficient to copy just the user CA certificate to the host machines and the host CA certificate to the user machines. All the users’ certificates need not be copied to all the host machines and all the host machines' certificates need not be copied to all the user machines.

SSH certificates have an expiry date and validity period stamped in them. So, all SSH certificates will eventually expire sometime in the future. The need to rotate SSH certificates periodically is built into the technology.

An organization’s security policy could decide the lifetime of a certificate issued to its hosts and users.

SSH certificates can be issued to a specific identifiable entity within an organization. So SSH certificates have the host or user name stamped in them to identify who owns the certificate. Role based access control (RBAC) is possible with SSH certificates.

When a user connects to a server for the first time, no weird cryptic messages will be displayed on the screen to identify the fingerprint and trust the host key. The authenticity of the host the user is trying to connect to will be automatically verified using the Host CA Certificate stored locally in the user’s machine.

Drawbacks of SSH certificate based authentication:

Though, SSH certificate based authentication addresses most of the security problems in the SSH public private key based authentication, it still has the following caveats in it:

  1. SSH private keys need to be safely copied to the corresponding host or user machines without leaving any traces of it anywhere.
  2. Rotating or renewing certificates takes time and effort to distribute them. Anyone who gets hold of an SSH certificate could login to a host without any issues. Short-lived user certificates are highly secure because it reduces the duration of the damage any unwanted user who holds such a user certificate could cause. However, short-lived certificates require more frequent rotations and therefore more effort from the admin teams. Long-lived certificates have the same inherent security risk as the SSH public private key based authentication but requires reduced effort from the admin teams.

BastionXP Bastion Host Solution:

BastionXP Bastion Host solution has a built-in PKI and CA that uses Single-Sign On (SSO) with Multi-Factor Authentication (MFA) to automatically generate and distribute short-lived user SSH certificates to end user machines. User SSH certificates expire in 8 hours by default and can be configured to suit an organization’s security policy, to any value like 4 weeks or 30 secs or so.

Once users use the SSH certificate to login to a host, they can continue with that SSH session for any longer even if the certificate expires. But, if they exit from the SSH session and need re-login, the users need to SSO login again using their MFA credentials to generate a new SSH certificate.

Because user SSH certificates are short-lived, no certificate cleaning or housekeeping is required in the host or user machines, when a user leaves an organization or when teams get dismantled. The user certificate becomes invalid the next day.

Components of the BastionXP Bastion Host Solution:

BastionXP Bastion Host solution has three components:

  • SSH Bastion Host with PKI/CA
  • BastionXP SSH Server,
  • BastionXP SSH client/authentication utility

Note: BastionXP Bastion Host solution can be configured to work as a pure play solution or work along with OpenSSH server and OpenSSH client software.

The BastionXP bastion host has a builtin auth server which does the PKI/CA functionality. It can also perform SSH proxy or Jump Host functionality to connect to hosts that exist safely behind it, protected from the internet.

The BastionXP SSH server (sshd) implementation runs on each host server, talks to the auth server in the BastionXP bastion host, downloads a host certificate from the auth server to the host and has the ability to record SSH sessions for auditing and playback purposes.

The BastionXP SSH client (a.k.a authentication CLI utility) allows users to authenticate using SSO and MFA based authentication mechanism (Okta/MS 365/G-Suite) to download a short-lived SSH certificate from the auth server(CA ) to the user’s laptop or desktop. The BastionXP SSH client also does the regular SSH client functionalities such as SSH sessions with hosts, agent forwarding, connect to target hosts via a proxy jump host etc.

BastionXP Bastion Host solution can be used in any brownfield deployments where OpenSSH servers and clients are already in use and is preferred over the BastionXP SSH server and client software. In such a scenario, BastionXP Bastion Host’s SSH proxy (Jump Host) functionality can be disabled and BastionXP would simply run in the PKI/CA only mode.

BastionXP Bastion Host solution could be deployed in three different ways. We’ll discuss each one of them in detail below.

BastionXP Bastion Host Deployment Option #1 - CA Only Mode

In the CA only mode of deployment, BastionXP Bastion Host will function only as a CA server to manage the SSH certificates. BastionXP Bastion Host’s SSH proxy mode (or Jump Host) functionality will be disabled. An OpenSSH server (already in place) will perform the Jump Host functionality.

SSH host certificates need to be generated by an admin using a BastionXP provided API executed from the host machines running OpenSSH server. Usually, SSH host certificates have a very long validity period. For example: 4 weeks, 26 weeks, or 52 weeks. This is a configurable value and can be set according to an organization’s security policy.

Users need to download, install and use a simple CLI tool provided by BastionXP to login into their SSO provider using MFA and download the short-lived user SSH certificates from the BastionXP CA to their laptops or a desktops. Users need to use this CLI tool only once everyday when they begin their work in the morning because the short-lived user certificate will be valid for 8 hours by default (This validity period is a configurable value).

BastionXP Bastion Host Deployment Option #2 - SSH Jump Host + CA Mode

In this mode, BastionXP Bastion Host will replace an OpenSSH server functioning as a Bastion Host or Jump Host. BastionXP Bastion Host will function as the SSH proxy server or Jump Host, in addition to performing the CA functionality for SSH certificate management. However, the hosts residing behind the Jump Host will still run the OpenSSH server and the user machines will continue to use the OpenSSH client. BastionXP Bastion Host can interoperate seamlessly with OpenSSH servers running in the host machines and the OpenSSH clients running the user machines.

BastionXP Bastion Host Option #3 - Pure Play Mode

In the pure play mode, BastionXP software will run everywhere in the network as an SSH server, SSH client and as an SSH Jump Host/CA server. The diagram below explains this mode of operation.

BastionXP Bastion Host Solution with SSH Certificate Management

Conclusion:

SSH public private key based authentication is prone for security breaches as it doesn’t have an expiry date and user or host identity associated with the keys. SSH certificate based authentication addresses several drawbacks of SSH public private key based authentication. BastionXP Bastion Host solution automates and simplifies the SSH certificate management while offering additional security services such as short-lived SSH keys, SSH session recording, monitoring, logging and tracking functionalities for auditing purposes

Free Trial

Start your free trial of BastionXP Bastion Host solution here.

To know more about BastionXP Bastion Host solution and request for a demo, please contact us at: [email protected]