SSH Certificates: How Do OpenSSH Certificates Compare to X.509?

X.509 is the first thing that comes to mind when discussing digital certificates. After all, it is the most widely used digital certificate in the PKI ecosystem and is the core component of SSL/TLS protocols, the technology that powers HTTPS. X.509 was first released on 25 November 1988 and is powerful, extensible and widely supported. But it’s not the only certificate format available out there. For example, the popular email encryption program PGP uses a custom certificate format instead of X.509. And PGP is not the only one to avoid X.509 — the most widely used remote access server OpenSSH also uses a custom certificate format. They call it certkeys!

OpenSSH has supported certificate-based authentication since version 5.4, but despite X.509 being widely supported, OpenSSH did not base the certificates in X.509 format but instead invented a simple but capable and compatible enough format for certificate-based authentication. Why? It is questionable, but the reasons must have been well-founded for sure!

In this blog post, we will explore how OpenSSH’s certificate compares with X.509, including a few strengths and weaknesses of the OpenSSH implementation.

Why OpenSSH implements a custom format instead of X.509?

The implementation details of OpenSSH certificate-based authentication are well-described in the PROTOCOL.certkeys document, but I’ll highlight the important points below:

Simple extension to existing public-key authentication

Let’s first try to understand how OpenSSH implements certificate-based authentication. In OpenSSH, the CA certificate is just public and private key pairs with additional identity and constraints data. The private key of the CA is used to sign user and host (SSH server) certificates. Once the keys are signed, they are distributed to users and hosts, respectively. The public key of the CA is copied to the SSH server, which is used to verify the user’s certificate.

For user authentication, the SSH client presents the user certificate (signed by CA) to the SSH server in each new SSH connection. The SSH server validates the certificate by checking it against the CA’s public key. Along with the signature validation, the server will also check if the user certificate is not expired and if it violates security constraints. Access is granted upon successful validation of the certificate.

This authentication flow is similar to public-key authentication with just additional verification of trust based on a signature signed by a trusted CA and additional constraints of expiry dates and identity attributes. Under the hood, this logical extension means that implementation for certificate-based authentication became as simple as extending the public-key authentication protocol as specified in RFC 4252. This is done via SSH protocol extension and adding new keys for certificates including following key types:

Reduced attack surface

Keeping the certificate format minimal helps reduce the attack surface. X.509 is a behemoth when it comes to certificate attributes and extensibility. Supporting X.509 certificates means implementing full-fledged X.509 PKI. Given X.509 is designed to be universal, as such it carries loads of features that simply may not be required for SSH authentication. Reducing attack surfaces is always a welcomed approach for maintaining high security.

Reduced dependencies on external crypto libraries

This is not explained in the design document shared above, but I’ve seen this mentioned many times in discussion forums. Historically, OpenSSH crypto libraries were all based on OpenSSL’s crypto library, and as such, any feature sets on the OpenSSH protocol should be first supported by OpenSSL. Implementing a custom format allows OpenSSH to maintain and support better solutions without the need to wait for implementations upstream.

Enforce better security by default

Custom implementations allow OpenSSH to support best practices for CA and certificate management. Maintaining separate CA for a user and host authentication is an expected workflow in OpenSSH. Algorithms such as AES, ChaCha20, RSA, ECDSA, and Ed25519 are readily available.

Limitations of OpenSSH certificates over X.509

Simplicity comes with a price! OpenSSH certificates lack the broader customizability that X.509 supports.

One additional Certificate Authority (CA) to maintain

X.509 are widely supported, so chances are that your organization already maintains trust with an internal or external X.509 CA. But to support certificate-based authentication, you will need to maintain another CA that can handle SSH certificate signing and issuance.

Misses X.509 features such as cross-signing and CA hierarchy

OpenSSH has no mechanism to support cross-signing CA and managing multiple CA hierarchies besides the notion of user and host CA certificates. This can pose a challenge in a large-scale implementation where multiple intermediate CA can come in handy during CA rotation or CA migration.

Comparing X.509 properties with OpenSSH certificate

Below is a high-level overview of X.509 and OpenSSH certificate properties.

Let’s look at a sample OpenSSH certificate created using OpenSSH ssh-keygen below:

Let’s look at a sample X.509 certificate created using OpenSSL below:

Conclusion

In this post, we explained why OpenSSH implements its own certificate format and how it differs from X.509 standard. The OpenSSH implementation is simple enough to cater to the requirements of certificate-based authentication as implemented by OpenSSH. But maintaining two different types of CA can be challenging for organizations. Should OpenSSH ultimately support X.509? I like the fact that it’s simple yet secure enough with a reduced attack surface.

Automate X.509 and OpenSSH CA management with Remoteler

Remoteler implements both X.509 and OpenSSH CA management which makes it easy to manage certificate-based authentication in both small and large-scale infrastructure deployments.

Learn how Remoteler works!