Lip Colour Finder – Control Through Systemd

I recently configured Lip Colour Finder to utilize systemd to manage its components. This post will be a small nugget on that process as opposed to the usual deep-dive. Systemd is a software suite that can be used for the management of system processes. I turned to it as I wanted to achieve the following goals:

• Run the Lip Colour Finder (LCF) application with a user less privileged than root by default.
• Automatically restart the LCF application should its processes stop.
• Automatically start the LCF application on reboot of the server.

Previously I started gunicorn as a daemon (from root) manually with the required configuration options. This has worked adequately up to now as the LCF application is quite stable and does not currently experience runtime errors (knock on wood). Gunicorn also automatically restarts workers whenever there is a problem responding to a request. I was worried however about unknown errors causing the application to crash when I was not there to monitor the application and restart it. I was also worried about the security risk of my application being compromised and the attacker gaining root access to the server through it.

Using this guide provided by DigitalOcean I created the following systemd unit file:

[Unit]
Description=gunicorn daemon for LCFS Django Project
Before=nginx.service
After=network.target

[Service]
EnvironmentFile=/home/lcfs/lcfs_env
WorkingDirectory=/var/www/lcfs/lip_colour_finder_site
ExecStart=/home/lcfs/virtualenvs/lcfs/bin/gunicorn --name=lcfs --pythonpath=/var/www/lcfs/lip_colour_finder_site --bind unix:/home/lcfs/gunicorn.socket --config /etc/gunicorn.d/gunicorn.py lip_colour_finder_site.wsgi:application
Restart=always
SyslogIdentifier=gunicorn
User=lcfs
Group=lcfs

[Install]
WantedBy=multi-user.target

One thing that the guide did not cover was the ‘EnvironmentFile’ directive. I keep sensitive information out of version control and load them at runtime from environmental variables. Previously I kept them in root’s .bashrc but systemd required a different format. Variables had to be listed in the form:

Variable_Name=Variable

instead of

Variable_Name='Variable'

In the unit file you can see that the service is configured to be restarted whenever it stops and to be run as the less-privileged ‘lcfs’ user. I can also utilize my Python virtual environment by directly loading the gunicorn installation contained within it.

With a simple:

systemctl restart gunicorn

I can now run the server. I can also check on its status with:

systemctl status gunicorn

and receive the following output:

lcfs@lcfs-conch:~$ systemctl status gunicorn
● gunicorn.service - gunicorn daemon for LCFS Django Project
Loaded: loaded (/etc/systemd/system/gunicorn.service; enabled; vendor preset: enabled)
Active: active (running) since Tue 2019-01-15 08:59:57 EST; 1h 50min ago
Main PID: 11140 (gunicorn)
Tasks: 4 (limit: 2361)

The LCF application is now managed by systemd and my confidence in the application’s resilience has increased. The next step will be to add status monitoring with notifications.

P.S. I previously used TCP/IP sockets for communication between NGINX and gunicorn. While switching to systemd for management I decided to also switch to lighter Unix sockets as all communication between the two applications currently occurs on the same machine. I used the following Python code to create the Unix socket referenced in the systemd unit file above:

import socket as s

sock = s.socket(s.AF_UNIX)
sock.bind('/home/lcfs/gunicorn.socket')