Side Quest: Using Let's Encrypt with AWS Route53

I use AWS Route53 for DNS for one of my domains. While setting up Let’s Encrypt certificates on a reverse proxy, it occurred to me that I could share my opinions on managing credentials for this type of operation. Sure, no one asked and it’s entirely possible that no one cares, but it’s my blog and I’ll do it anyway. So there.

Getting Started

All sorts of information exists in the wild on how to set up credentials to make certbot or acme.sh work for Let’s Encrypt certificates. That’s all well and good, but I’m an SRE and we tend to think about “what could happen if something goes wrong?” The main thing I can think of going wrong: the credentials get compromised. How can we limit the effort required to remedy the situation?

I use the approach below to limit the blast radius if something goes wrong. I create a separate user for every system using a Let’s Encrypt client. If I lose the credentials or they become compromised, I only have to rotate credentials on that one system. Had I reused the same account everywhere, then I would have to scramble to update credentials everywhere.

I could create separate users that attach the policy directly, but then it becomes more of a chore to audit which user accounts have this access. Doing it with groups means all I have to do is see which users are members of the group and Bob’s your uncle. I suppose I could look at the policy and see which entities use it, but this seems cleaner to me.

My Solution

This approach requires a bit more work up front, but limits the blast radius when things go wrong. I considered including screen shots for some of the navigation, but the AWS console changes every so often and I didn’t want to be misleading (really I just couldn’t be bothered). All of the below can be done using the aws cli or a variety of other ways, none of which I’ll cover here.

Create a policy

First up, create your own policy in IAM. In the AWS console, go to IAM –> Policies –> Create policy. Enter the following into the JSON tab:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "route53:GetHostedZone",
                "route53:ListHostedZones",
                "route53:ListHostedZonesByName",
                "route53:GetHostedZoneCount",
                "route53:ChangeResourceRecordSets",
                "route53:ListResourceRecordSets",
                "route53:GetChange"
            ],
            "Resource": "*"
        }
    ]
}

This policy has the necessary permissions for either acme.sh or certbot which have slightly different permission requirements. Other Let’s Encrypt clients may have permissions requirements not listed above, so modify as needed.

Create a user group

Now that you have the policy, create a user group that uses it. In the AWS console go to IAM –> User groups –> Create group. While creating the group, attach the permissions policy that you created above. Gee, that was easy.

Create a user

Finally, create a user. In the AWS console, go to IAM –> Users –> Create user. For credential type, select “Access key - Programmatic access”. When setting permissions, add the user to the group created above. When you’re done you will have your only chance to save the access key ID & secret access key so make sure to do so and save them somewhere secure!

All Done!

Setting up that new nginx reverse proxy in a jail on my TrueNAS server reminded me how easy it is to use this setup once in place. I simply created a new user in the group and I was done. That let me tinker with the reverse proxy without risk of breaking the certificate renewal workflow for the TrueNAS console itself.