Devops Discoveries

X509 Certificates in Salt, implementation details

Saltstack has recently accepted my pull request and integrated the x509 module I’ve spent the last few weeks working on. Most of it’s functionality including a complete PKI example is explained in the documentation, this blog post is to go over some more of the details of how and why I made it for anyone who may not want to just read the source code.

I started out with a need to manage X509 certificates for geographically distributed linux based VPN routers. I thought this would be pretty easy with the built in tls module, but this didn’t meet my needs for a few reasons. For one thing the module is stateful. It requires having a specific directory structure setup and expects files to be in specific places. I don’t really like this idea, I prefer my stateful configuration to be in my state files, and my execution modules to get everything they need via arguments. Secondly this module has no methods for having a CA sign certificates for remote servers.

What I wanted in the end was to be able to configure a state which would generate a private key on my VPN router, submit the corresponding public key back to my CA server, which would then generate a certificate with that public key and give it back to the VPN router. And with the great tools provided by salt I was able to do this.

This module and state does require some understanding of how X509 certificates work. For example, this state does not have the concept of a ‘CA’ per say. Because in reality a CA is the same as any other private key and certificate. The only difference is what extensions are included in the certificate. So to generate a CA using this module, you first generate a private key, then generate a certificate with the correct extensions, most importantly basicConstraints: CA:true. If you pass in a public_key argument that is the corresponding key to the signing_private_key argument, this will create a self-signed certificate. A self-signed certificate with CA: true is a CA.

One thing to keep in mind when looking at this state is that some times you will see in the examples passing a private key as a public key to a module. If you’re concerned about this, congratulations, you understand the importance of keeping private keys private. But this functionality is safe to use. Any property passed into the module as a public key will first be run through the get_public_key function. This means if you pass in a private key, the matching public key will be generated and used. This happens before anything is sent over the network, so you can be sure that the CA you are having sign your certificate will never have access to your private key.

When you’re ready to create a certificate signed by the remote CA, you use the create_certificate state again, but instead of using the signing_private_key argument, use the ca_server argument. When using ca_server you must necessarily include the signing_policy argument. Signing policies are properties defined in the minion configuration of the CA server. They define mandatory certificate properties and which minion IDs are allowed to get certificates signed. For example, by including the basicConstraints extension in the signing policy, the CA will be protected against issuing certificates that can be used to sign other certificates. When this state is run, the public key, along with any other properties for the certificate are sent across the salt event bus using peer communication. The CA server checks the minion id that sent the request to determine if it has permission to get a certificate using the requested signing_policy. If so it creates the certificate, overriding any requested properties with those specified in the signing_policy, then returns the signed certificate back to the minion that requested it.

One more very useful property of controlling these in states, is the days_remaining property. The state will check the expiration date of a certificate when the state runs and if days_remaining is configured, it will automatically renew the certificate when fewer than that number of days are remaining. This can greatly improve security of a PKI infrastructure by allowing self-manged short-lived certificates. I can have each VPN router generate certificates which the signing policy limits to only being valid for 2 weeks. The VPN server’s state can automatically request a new certificate when there are less than 7 days remaining. Through the use of prereqs and the new: True parameter on the private key state I can also make sure that new private keys are generated for every certificate renewal. This means that if a VPN router’s private key is ever compromised, it is only useful for at most 2 weeks.