Get acme-client running on Ubuntu

I finally found a way to sort of securely start using Let’s Encrypt on my servers. One of the reasons I have been hesitant about installing the official client, certbot, is because of some statements made by Theo de Raadt about the security of most clients of the ACME protocol (which is the protocol to talk to Let’s Encrypt). Installing certbot on Ubuntu 16.04 means adding more than 600.000 lines of Python and more than 400.000 lines of C to your installation, and that’s even without counting OpenSSL. Furthermore this code is meant to run periodically as root and touches the network. Oh the horror!

Of course with some extra effort it is possible to run certbot as a dedicated user with minimal privileges. But instead of using a monolithic program, I’d like to focus my energy on a tool that is made with security by design. One that nicely separates all the important parts from each other like code that touches the keys, from code that touches the file system. And code that touches the network, from the complex code that parses the X.509 certificates. Hence, this tutorial will be about installing acme-client (formerly known as letskencrypt), which is made by Kristaps Dzonsons.

Note: the major drawback about the solution presented here, is that you have to watch for updates yourself of both LibreSSL and acme-client. If there is an update to any of those, you have to redo the steps in this tutorial in order to apply the patches. Be warned.

Bring out the tools

First make sure you have a working compiler and package configuration tool installed. Furthermore ensure all required libraries for building the software are available. These tools are only needed in order to compile new versions of the software and are not required for running them.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
tim@ubuntu:~$ sudo apt-get install build-essential pkg-config libbsd-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
binutils cpp cpp-5 dpkg-dev fakeroot g++ g++-5 gcc gcc-5 gcc-5-base libalgorithm-diff-perl libalgorithm-diff-xs-perl libalgorithm-merge-perl libasan2 libatomic1 libc-dev-bin
libc6 libc6-dev libcc1-0 libcilkrts5 libdpkg-perl libfakeroot libfile-fcntllock-perl libgcc-5-dev libgomp1 libisl15 libitm1 liblsan0 libmpc3 libmpx0 libquadmath0
libstdc++-5-dev libstdc++6 libtsan0 libubsan0 linux-libc-dev make manpages-dev
Suggested packages:
binutils-doc cpp-doc gcc-5-locales debian-keyring g++-multilib g++-5-multilib gcc-5-doc libstdc++6-5-dbg gcc-multilib autoconf automake libtool flex bison gdb gcc-doc
gcc-5-multilib libgcc1-dbg libgomp1-dbg libitm1-dbg libatomic1-dbg libasan2-dbg liblsan0-dbg libtsan0-dbg libubsan0-dbg libcilkrts5-dbg libmpx0-dbg libquadmath0-dbg glibc-doc
libstdc++-5-doc make-doc
The following NEW packages will be installed:
binutils build-essential cpp cpp-5 dpkg-dev fakeroot g++ g++-5 gcc gcc-5 libalgorithm-diff-perl libalgorithm-diff-xs-perl libalgorithm-merge-perl libasan2 libatomic1 libbsd-dev
libc-dev-bin libc6-dev libcc1-0 libcilkrts5 libdpkg-perl libfakeroot libfile-fcntllock-perl libgcc-5-dev libgomp1 libisl15 libitm1 liblsan0 libmpc3 libmpx0 libquadmath0
libstdc++-5-dev libtsan0 libubsan0 linux-libc-dev make manpages-dev pkg-config
The following packages will be upgraded:
gcc-5-base libc6 libstdc++6
3 upgraded, 38 newly installed, 0 to remove and 103 not upgraded.
Need to get 41.5 MB of archives.
After this operation, 144 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 libc6 amd64 2.23-0ubuntu5 [2,589 kB]
Get:2 http://us.archive.ubuntu.com/ubuntu xenial/main amd64 libmpc3 amd64 1.0.3-1 [39.7 kB]
... lots and lots of output ...
Setting up pkg-config (0.29.1-0ubuntu1) ...
Processing triggers for libc-bin (2.23-0ubuntu3) ...

Creating acme-client

Now that all the tools are in place, fetch acme-client and since acme-client depends on the libtls api, fetch LibreSSL as well.

1
2
3
4
5
tim@ubuntu:~$ wget -q https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-2.5.0.tar.gz
tim@ubuntu:~$ wget -q https://kristaps.bsd.lv/acme-client/snapshots/acme-client-portable.tgz
tim@ubuntu:~$ sha256sum libressl-2.5.0.tar.gz acme-client-portable.tgz
8652bf6b55ab51fb37b686a3f604a2643e0e8fde2c56e6a936027d12afda6eae libressl-2.5.0.tar.gz
910f4ffab4aea2dc9563405aa6a53e85d00166a020c74c28d719f290c610e71e acme-client-portable.tgz

There are different ways to verify your download, but all involve a trust anchor. Check with the LibreSSL community, like consulting the OpenBSD mirrors and mailing list archives to obtain valid hashes. You ideally do this from different computers using different networks. Never trust a single blog like this one. Furthermore you can Google on the checksums of the files you just downloaded and make sure that they show up at different reputable sites with dates long before now.

Assuming you have a trusted copy of LibreSSL, let’s compile the sources. Because we don’t want to install LibreSSL on the system and mix it up with the installed version of OpenSSL, we’re going to keep it in it’s own directory.

1
2
3
4
5
6
7
8
9
tim@ubuntu:~$ tar zxf libressl-2.5.0.tar.gz
tim@ubuntu:~$ LSSLP=$(pwd)/libressl-2.5.0 # save a pointer to this dir
tim@ubuntu:~$ cd $LSSLP
tim@ubuntu:~/libressl-2.5.0$ ./configure
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
... lots and lots of output ...
config.status: executing depfiles commands
config.status: executing libtool commands
1
2
3
4
5
6
tim@ubuntu:~/libressl-2.5.0$ make
Making all in crypto
make[1]: Entering directory '/home/tim/libressl-2.5.0/crypto'
... lots and lots of output ...
make[1]: Nothing to be done for 'all-am'.
make[1]: Leaving directory '/home/tim/libressl-2.5.0'

Remove the shared library files so that these sources can be deleted once the acme-client binary is linked.

1
tim@ubuntu:~/libressl-2.5.0$ rm {ssl,tls,crypto}/.libs/*.so*

Now you have a fresh new copy of LibreSSL which is ready to be used. Let’s move on to the final part, compiling acme-client. Luckily, this is not as time consuming as compiling LibreSSL.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
tim@ubuntu:~/libressl-2.5.0$ cd ..
tim@ubuntu:~$ tar zxf acme-client-portable.tgz
tim@ubuntu:~$ cd acme-client-portable-0.1.15/
tim@ubuntu:~/acme-client-portable-0.1.15$ CFLAGS="-I$LSSLP/include" LDFLAGS="-L$LSSLP/ssl/.libs -L$LSSLP/tls/.libs -L$LSSLP/crypto/.libs" make
cc -I/home/tim/libressl-2.5.0/include -g -W -Wall -DHAVE_CONFIG_H -I/usr/local/include/libressl -c -o acctproc.o acctproc.c
cc -I/home/tim/libressl-2.5.0/include -g -W -Wall -DHAVE_CONFIG_H -I/usr/local/include/libressl -c -o base64.o base64.c
cc -I/home/tim/libressl-2.5.0/include -g -W -Wall -DHAVE_CONFIG_H -I/usr/local/include/libressl -c -o certproc.o certproc.c
cc -I/home/tim/libressl-2.5.0/include -g -W -Wall -DHAVE_CONFIG_H -I/usr/local/include/libressl -c -o chngproc.o chngproc.c
cc -I/home/tim/libressl-2.5.0/include -g -W -Wall -DHAVE_CONFIG_H -I/usr/local/include/libressl -c -o dbg.o dbg.c
cc -I/home/tim/libressl-2.5.0/include -g -W -Wall -DHAVE_CONFIG_H -I/usr/local/include/libressl -c -o dnsproc.o dnsproc.c
cc -I/home/tim/libressl-2.5.0/include -g -W -Wall -DHAVE_CONFIG_H -I/usr/local/include/libressl -c -o fileproc.o fileproc.c
cc -I/home/tim/libressl-2.5.0/include -g -W -Wall -DHAVE_CONFIG_H -I/usr/local/include/libressl -c -o http.o http.c
cc -I/home/tim/libressl-2.5.0/include -g -W -Wall -DHAVE_CONFIG_H -I/usr/local/include/libressl -c -o jsmn.o jsmn.c
cc -I/home/tim/libressl-2.5.0/include -g -W -Wall -DHAVE_CONFIG_H -I/usr/local/include/libressl -c -o json.o json.c
cc -I/home/tim/libressl-2.5.0/include -g -W -Wall -DHAVE_CONFIG_H -I/usr/local/include/libressl -c -o keyproc.o keyproc.c
cc -I/home/tim/libressl-2.5.0/include -g -W -Wall -DHAVE_CONFIG_H -I/usr/local/include/libressl -c -o main.o main.c
cc -I/home/tim/libressl-2.5.0/include -g -W -Wall -DHAVE_CONFIG_H -I/usr/local/include/libressl -c -o netproc.o netproc.c
cc -I/home/tim/libressl-2.5.0/include -g -W -Wall -DHAVE_CONFIG_H -I/usr/local/include/libressl -c -o revokeproc.o revokeproc.c
cc -I/home/tim/libressl-2.5.0/include -g -W -Wall -DHAVE_CONFIG_H -I/usr/local/include/libressl -c -o rsa.o rsa.c
cc -I/home/tim/libressl-2.5.0/include -g -W -Wall -DHAVE_CONFIG_H -I/usr/local/include/libressl -c -o util.o util.c
cc -I/home/tim/libressl-2.5.0/include -g -W -Wall -DHAVE_CONFIG_H -I/usr/local/include/libressl -c -o util-portable.o util-portable.c
cc -I/home/tim/libressl-2.5.0/include -g -W -Wall -DHAVE_CONFIG_H -I/usr/local/include/libressl -c -o sandbox-null.o sandbox-null.c
cc -o acme-client acctproc.o base64.o certproc.o chngproc.o dbg.o dnsproc.o fileproc.o http.o jsmn.o json.o keyproc.o main.o netproc.o revokeproc.o rsa.o util.o util-portable.o sandbox-null.o -L/home/tim/libressl-2.5.0/ssl/.libs -L/home/tim/libressl-2.5.0/tls/.libs -L/home/tim/libressl-2.5.0/crypto/.libs -L/usr/local/lib -ltls -lssl -lcrypto -lbsd

Install the new acme-client binary:

1
2
3
4
5
tim@ubuntu:~/acme-client-portable-0.1.15$ sudo make install
mkdir -p /usr/local/man/man1
mkdir -p /usr/local/bin
install -m 0755 acme-client /usr/local/bin
install -m 0644 acme-client.1 /usr/local/man/man1

Symlink the trust CA store to a location acme-client expects and setup the base directories that will later hold the keys, certificates and chrooted processes.

1
2
3
4
tim@ubuntu:~/acme-client-portable-0.1.15$ cd ..
tim@ubuntu:~$ sudo ln -s certs/ca-certificates.crt /etc/ssl/cert.pem
tim@ubuntu:~$ sudo mkdir -pm700 /etc/ssl/acme/private /etc/acme
tim@ubuntu:~$ sudo mkdir /var/empty

Using acme-client

With the binary in place, we’re ready to start requesting a new certificate. Say we’ve got Nginx running and it serves blog.netsend.nl, which is stored in /var/www/blog.netsend.nl/public/. The document root of your site, in this example /var/www /blog.netsend.nl/public, should contain a .well-known/acme-challenge directory.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
tim@ubuntu:~$ sudo mkdir -p /var/www/blog.netsend.nl/public/.well-known/acme-challenge
tim@ubuntu:~$ sudo acme-client -vmNnC /var/www/blog.netsend.nl/public/.well-known/acme-challenge blog.netsend.nl
acme-client: /etc/ssl/acme/blog.netsend.nl: creating directory
acme-client: /etc/ssl/acme/private/blog.netsend.nl: creating directory
acme-client: /etc/acme/blog.netsend.nl: creating directory
acme-client: /etc/ssl/acme/private/blog.netsend.nl/privkey.pem: generated RSA domain key
acme-client: /etc/acme/blog.netsend.nl/privkey.pem: generated RSA account key
acme-client: https://acme-v01.api.letsencrypt.org/directory: directories
acme-client: acme-v01.api.letsencrypt.org: DNS: 23.62.131.169
acme-client: acme-v01.api.letsencrypt.org: DNS: 2a02:26f0:f4:2a4::3d5
acme-client: acme-v01.api.letsencrypt.org: DNS: 2a02:26f0:f4:280::3d5
acme-client: https://acme-v01.api.letsencrypt.org/acme/new-reg: new-reg
acme-client: https://acme-v01.api.letsencrypt.org/acme/new-authz: req-auth: blog.netsend.nl
acme-client: /var/www/blog.netsend.nl/public/.well-known/acme-challenge/H9SYp8xTI3kHvO_0Rm02mDXYjuhZ8kFcDm6YdbWZyFk: created
acme-client: https://acme-v01.api.letsencrypt.org/acme/challenge/k8ZqX5xydy-AuoTIf78j3nqbOwGvJnoUYQlkX9HO2Zw/482873557: challenge
acme-client: https://acme-v01.api.letsencrypt.org/acme/challenge/k8ZqX5xydy-AuoTIf78j3nqbOwGvJnoUYQlkX9HO2Zw/482873557: status
acme-client: https://acme-v01.api.letsencrypt.org/acme/new-cert: certificate
acme-client: http://cert.int-x3.letsencrypt.org/: full chain
acme-client: cert.int-x3.letsencrypt.org: DNS: 2a02:26f0:60::173e:620b
acme-client: cert.int-x3.letsencrypt.org: DNS: 2a02:26f0:60::173e:624b
acme-client: cert.int-x3.letsencrypt.org: DNS: 95.101.39.43
acme-client: cert.int-x3.letsencrypt.org: DNS: 95.101.39.58
acme-client: /etc/ssl/acme/blog.netsend.nl/chain.pem: created
acme-client: /etc/ssl/acme/blog.netsend.nl/cert.pem: created
acme-client: /etc/ssl/acme/blog.netsend.nl/fullchain.pem: created

Now configure your webserver to use the new certificate which is stored in /etc/ssl/acme/blog.netsend.nl/fullchain.pem and the key which is stored in /etc/ssl/acme/private/blog.netsend.nl/privkey.pem. See the documentation of your webserver for specific instructions. In Nginx this would be:

1
2
ssl_certificate /etc/ssl/acme/blog.netsend.nl/fullchain.pem;
ssl_certificate_key /etc/ssl/acme/private/blog.netsend.nl/privkey.pem;

Restart your webserver and see if the new certificate is successfully served. Tip: use https://www.ssllabs.com/ssltest/ to test your SSL configuration.

The last step is to automatically check for renewal via a cronjob, this can be as simple as:

1
echo '50 11 * * * root acme-client -mC /var/www/blog.netsend.nl/public/.well-known/acme-challenge blog.netsend.nl' >> /etc/crontab

And we’re all set! Now if there is any security update for either LibreSSL or acme-client, fetch the new version, verify your downloads and compile and install the new version.

Please contact me for any suggestions or criticism.

Chroot NSD on Ubuntu Server 14.04

We’re going to mitigate some risks that are inherent when running a server by jailing our nsd authoritative name server into it’s own chroot and making sure it runs with the least privileges possible. This manual is split into two parts. The first part is what it’s all about when it comes to jailing your daemon. Luckily this is not a lot of work because nsd has builtin support for chroots. The second part serves as an example of where to put your zone files and how to include them in your configuration.

  1. Setup chroot
  2. Setup zone example.com

All commands should be executed as root.

1. Setup chroot

We’ll assume you have nsd installed, if not run apt-get install nsd.

Before we lock up the daemon into the jail we have to make sure it has some space where it can write out it’s thoughts. We’ll put the zone files in a different directory just to keep things clear.

1
2
3
4
# mkdir -m 775 /etc/nsd/db /etc/nsd/run /etc/nsd/run/xfr
# chgrp -R nsd /etc/nsd/db /etc/nsd/run
# chmod o-rwx /etc/nsd/run/xfr
# mkdir -m 755 /etc/nsd/zones

Create a logging socket in the chroot so that any complaints the daemon might have can be heard.

1
2
3
# mkdir -m 755 /etc/nsd/dev
# echo '$AddUnixListenSocket /etc/nsd/dev/log' > /etc/rsyslog.d/nsd.conf
# service rsyslog restart

Create a new config file in /etc/nsd/nsd.conf.d/local.conf that has pointers to the new writable locations and contains the instructions to chroot and drop privileges.

1
2
3
4
5
6
7
8
9
10
11
# cat > /etc/nsd/nsd.conf.d/local.conf <<EOF
server:
username: nsd
chroot: /etc/nsd
database: /etc/nsd/db/nsd.db
pidfile: /etc/nsd/run/nsd.pid
zonelistfile: /etc/nsd/db/zone.list
zonesdir: /etc/nsd/zones
xfrdfile: /etc/nsd/run/xfrd.state
xfrdir: /etc/nsd/run/xfr
EOF

Include the new config file and restart nsd.

1
2
# echo 'include: /etc/nsd/nsd.conf.d/local.conf' >> /etc/nsd/nsd.conf
# service nsd restart

At last pinch a hole in the firewall so it can communicate with the outside world.

1
# ufw allow from any to any port 53

Thanks to the fact that nsd has builtin support for chrooting this is all that comes to it.

2. Setup zone example.com

Now a chrooted authoritative name server without zones doesn’t make any sense, at all. I’ll show you how to link a zone by using an example. Say your name server is located at ns.example.net and it has to answer questions for example.com. Create a zone file for example.com in /etc/nsd/zones/example.com.zone.

1
2
3
4
5
6
7
# cat > /etc/nsd/zones/example.com.zone <<EOF
\$ORIGIN example.com.
\$TTL 86400
@ IN SOA ns.example.net. hostmaster.example.net. ( 2015033100 24h 2h 3W12h 2h20M )
IN NS ns.example.net.
www IN A 203.0.113.2
EOF

Add this zone to the configuration and restart nsd.

1
2
3
4
5
6
7
8
# cat >> /etc/nsd/nsd.conf.d/local.conf <<EOF

zone:
name: example.com
zonefile: example.com.zone
EOF

# service nsd restart

The new zone can be tested by using dig(1) to ask the name server the ip address of www.example.com.

1
2
$ dig @ns.example.net www.example.com +short
203.0.113.2

Well there you go. The name server at ns.example.net said that www.example.com is located at 203.0.113.2. You’ve chrooted the nsd daemon and setup an example zone. Once you’ve added all your zones you’re done.

Please contact me for any suggestions or criticism.