Auto-update of Letsencrypt Wildcard TLS Certificates

by Sebastien Mirolo on Tue, 16 Jul 2024

Letsencrypt started to issue wildcard certificates in . Verification of ownership is done through a TXT DNS record, which means for auto-updating, we need to be able to update DNS entries programmatically. Since the domain we are planning to use wildcard certificates on is managed by AWS Route53, this is perfect.

Using a up-to-date version of certbot

The certbot version installed on Amazon Linux 2 shows its age.

Terminal
$ cat /etc/os-release
...
PRETTY_NAME="Amazon Linux 2"
...

$ certbot --version
certbot 1.11.0

$ cat /usr/bin/certbot
#!/usr/bin/python2
...

The certbot executable installed by the AL2 package manager relies on python2, but the certbot-route53 plugin depends on boto3 (i.e. Python3). Since AL2 is coming to end-of-life in 2025, and we are already embarked into a migration from AL2 to AL2023, this post deals with renewing Letsencrypt wildcard TLS certificates on AL2023.

Terminal
$ cat /etc/os-release
...
PRETTY_NAME="Amazon Linux 2023"
...

$ certbot --version
certbot 2.6.0

$ cat /usr/bin/certbot
#!/usr/bin/python3
...

Installing certbot-route53 plugin

To renew Letsencrypt wildcard TLS certificates, we will need the certbot-route53 plugin.

Terminal
$ certbot certonly --test-cert --preferred-challenges=dns --dns-route53 -d *.example.com -d example.com
Error: The requested dns-route53 plugin does not appear to be installed

The error message is pretty explicit. The certbot-route53 plugin is not installed by default.

Terminal
$ dnf search certbot-route53
Amazon Linux 2023 repository                     35 MB/s |  25 MB     00:00
Amazon Linux 2023 Kernel Livepatch repository    53 kB/s |  11 kB     00:00
No matches found.

The certbot-route53 plugin is not available as a native package. We will need to install it through a Python package manager (pip).

Terminal
$ dnf install python3-pip
...
  Verifying        : python3-pip-21.3.1-2.amzn2023.0.7.noarch               2/2
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/dnf/plugin.py", line 104, in _caller
    getattr(plugin, method)()
  File "/usr/lib/python3.9/site-packages/dnf-plugins/post-transaction-actions.py", line 146, in transaction
    command = self._replace_vars(ts_item, a_command)
  File "/usr/lib/python3.9/site-packages/dnf-plugins/post-transaction-actions.py", line 104, in _replace_vars
    result = libdnf.conf.ConfigParser_substitute(command, vardict)
AttributeError: module 'libdnf.conf' has no attribute 'ConfigParser_substitute'

Installed:
  python3-pip-21.3.1-2.amzn2023.0.7.noarch

Complete!

$ sudo pip3 install certbot-route53
Installing collected packages: zipp, botocore, s3transfer, importlib-metadata, acme, certbot, boto3, certbot-dns-route53, certbot-route53
  Attempting uninstall: acme
    Found existing installation: acme 2.6.0
ERROR: Cannot uninstall acme 2.6.0, RECORD file not found. Hint: The package was installed by rpm.

Unfortunately we stumble upon two errors while installing certbot-route53 into the default /usr/lib/python3.9/site-packages directory.

We thus need to install the package in a local directory, and find a way to add this local install directory to the certbot plugin search path. Fortunately that works!

Terminal
$ python3 -m venv /usr/local
$ /usr/local/bin/pip install certbot-route53
$ export CERTBOT_PLUGIN_PATH=/usr/local/lib/python3.9/site-packages
$ certbot plugins
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
* dns-route53
Description: Obtain certificates using a DNS TXT record (if you are using AWS
Route53 for DNS).
Interfaces: Authenticator, Plugin
Entry point: EntryPoint(name='dns-route53',
value='certbot_dns_route53._internal.dns_route53:Authenticator',
group='certbot.plugins')
...

Updating AWS IAM profile

When the certbot-route53 plugin is installed and found properly, we are going to run in AWS Route53 permission issues running the certbot command until the IAM profile for the EC2 instance is updated.

Terminal
$ certbot certonly --test-cert --preferred-challenges=dns --dns-route53 -d *.example.com -d example.com
An error occurred (AccessDenied) when calling the ListHostedZones operation: User: AWS_IAM_PROFILE is not authorized to perform: route53:ListHostedZones because no identity-based policy allows the route53:ListHostedZones action

In our case, we had to add ListHostedZones and GetChange to the IAM profile.

JSON
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Statement1",
            "Effect": "Allow",
            "Action": [
                "route53:ListHostedZones",
                "route53:GetChange"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

Auto-renew script

When the command is successful, we get an enigmatic message.

Terminal
$ certbot certonly --test-cert --preferred-challenges=dns --dns-route53 -d *.example.com -d example.com
...
Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/example.com/fullchain.pem
Key is saved at:         /etc/letsencrypt/live/example.com/privkey.pem
This certificate expires on 2024-10-14.
These files will be updated when the certificate renews.
Certbot has set up a scheduled task to automatically renew this certificate in the background.

Where is that task? How do I turn it off?

We can't find it in either /etc/cron.*, nor in the systemd timers.

Terminal
$ systemctl list-timers

Maybe because we are running against the staging website?

Terminal
$ cat /etc/letsencrypt/live/example.com/fullchain.pem | openssl x509 -noout -issuer
issuer=C = US, O = (STAGING) Let's Encrypt, CN = (STAGING) False Fennel E6

Let's acquire a production TLS certificate then.

Terminal
$ certbot certonly --dns-route53 -d *.example.com -d example.com
...
Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/example.com/fullchain.pem
Key is saved at:         /etc/letsencrypt/live/example.com/privkey.pem
This certificate expires on 2024-10-14.
These files will be updated when the certificate renews.
Certbot has set up a scheduled task to automatically renew this certificate in the background.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
If you like Certbot, please consider supporting our work by:
 * Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
 * Donating to EFF:                    https://eff.org/donate-le
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

$ systemctl status certbot-renew.timer
○ certbot-renew.timer - This is the timer to set the schedule for automated renewals
     Loaded: loaded (/usr/lib/systemd/system/certbot-renew.timer; disabled; preset: disabled)
     Active: inactive (dead)
    Trigger: n/a
   Triggers: ● certbot-renew.service

It does not appear the background task is running after all, which is exactly what we wanted since we have our own renewal service and timer.

Et Voila!

More to read

If you are looking for related posts, Whitelabel Domain with TLS and PostgreSQL, encryption and AWS RDS instance are good reads.

More technical posts are also available on the DjaoDjin blog. For fellow entrepreneurs, business lessons learned running a subscription hosting platform are also available.

by Sebastien Mirolo on Tue, 16 Jul 2024


Receive news about DjaoDjin in your inbox.

Bring fully-featured SaaS products to production faster.

Follow us on