mix_systemd
This library generates a systemd unit file to manage an Elixir application.
At its heart, it's a mix task which reads information about the project from
mix.exs
plus optional library configuration in config/config.exs
and
generates systemd unit files using Eex templates.
The goal is that the project defaults will generate a good systemd unit file, and standard options support more specialized use cases. If you need more customization, you can check the local copy of the templates into source control and modify them (and patches are welcome).
It uses standard systemd functions and conventions to make your app a more "native" OS citizen, and takes advantage of systemd features to improve security and reliability.
While it can be used standalone, more advanced use cases require scripts from e.g. mix_deploy.
Installation
Add mix_systemd
to the list of dependencies in mix.exs
:
def deps do
[
{:mix_systemd, "~> 0.1.0"}
]
end
Usage
This library works similarly to Distillery.
The init
task copies template files into your project, then the generate
task uses them to create the output files.
Run this command to initialize templates under the rel/templates/systemd
directory:
MIX_ENV=prod systemd.init
Next, generate output files under _project/#{mix_env}/systemd/lib/systemd/system
.
MIX_ENV=prod mix systemd.generate
Configuration
The library gets standard information in mix.exs
, e.g. the app name and
version, then calculates default values for its configuration parameters.
You can override these parameters using settings in config/config.exs
, e.g.:
config :mix_systemd,
app_user: "app",
app_group: "app",
base_dir: "/opt",
env_vars: [
"REPLACE_OS_VARS=true",
]
The following sections describe configuration options.
See lib/mix/tasks/systemd.ex
for all the details.
If you need to make changes not supported by the config options, then you can
check the templates into source control from rel/templates/systemd
and make
your own changes (contributions welcome!).
Basics
app_name
: Elixir application name, an atom, from the app
field in the mix.exs
project.
version
: version
from mix.exs
project.
ext_name
: External name, used for files and directories.
Default is app_name
with underscores converted to "-".
service_name
: Name of the systemd service, default ext_name
.
base_dir
: Base directory where app files go, default /srv
to
follow conventions.
deploy_dir
: Directory where app files go, default #{base_dir}/#{ext_name}
app_user
: OS user account that the app runs under, default ext_name
.
app_group
: OS group account, default ext_name
.
Directories
Modern Linux defines a set of directories which apps use for common purposes, e.g. configuration or cache files. See https://www.freedesktop.org/software/systemd/man/systemd.exec.html#RuntimeDirectory=
This library defines these directories based on the app name, e.g. /etc/#{ext_name}
.
It only creates directories that the app uses, default runtime
(/run/#{ext_name}
)
and configuration
(/etc/#{ext_name}
). If your app uses other dirs, set them in the
dirs
var:
dirs: [
:runtime, # Needed for RELEASE_MUTABLE_DIR, runtime-environment or conform
:configuration, # Needed for Erlang cookie
# :cache, # App cache files which can be deleted
# :logs, # App external log files, not via journald
# :state, # App state persisted between runs
# :tmp, # App temp files
],
For security, we set permissions to 750, more restrictive than the
systemd defaults of 755. You can configure them with e.g. configuration_directory_mode
.
See the defaults in lib/mix/tasks/systemd.ex
.
More recent versions of systemd (after 235) will create these directories at start time based on the settings in the unit file. For earlier systemd versions, you need to create them beforehand using scripts in e.g. mix_deploy.
systemd_version
: Sets the systemd version on the target system, default 235.
This determines which systemd features the library will enable. If you are
targeting an older OS relese, you may need to change it. Here are the systemd
versions in common OS releases:
- CentOS 7: 219
- Ubuntu 16.04: 229
- Ubuntu 18.04: 237
Additional directories
The library assues a directory structure under deploy_dir
which allows it to handle multiple reases,
similar to Capistrano.
scripts_dir
: dir for deployment scripts which e.g. start and stop the unit, defaultbin
.current_dir
: dir where the current Erlang release is unpacked or referenced by symlink, defaultcurrent
.releases_dir
: dir where versioned releases may be unpacked, defaultreleases
.flags_dir
: dir for flag files to trigger restart, e.g. whenrestart_method
is:systemd_flag
, defaultflags
.
When using multiple releases and symlinks, the deployment process works like this:
Create a new directory for the release with a timestamp like
/srv/foo/releases/20181114T072116
.Upload the new release tarball to the server and unpack it to the release dir
Make a symlink from
/srv/foo/current
to the new release dir.Restart the app.
If you are only keeping a single version, then you would deploy it to
the /srv/foo/current
dir.
Environment vars
The library sets a few common env vars directly in the unit file:
PORT
:env_port
var, default 4000LANG
:env_lang
var, defaulten_US.UTF-8
MIX_ENV
:mix_env
var, defaultMix.env()
HOME
:home_dir
var, defaultdeploy_dir
RELEASE_MUTABLE_DIR
: defaultruntime_dir
, e.g./run/#{ext_name}
DEFAULT_COOKIE_FILE
: default#{configuration_dir}/erlang.cookie
, e.g./etc/#{ext_name}/erlang.cookie
CONFORM_CONF_PATH
: default/etc/#{ext_name}/#{app_name}.conf
. ifconform
istrue
You can set additional vars in the env_vars
config var, e.g.:
env_vars: [
"REPLACE_OS_VARS=true",
]
The unit file also attempts to read environment vars from a series of files:
etc/environment
within the release, e.g./srv/app/currrent/etc/environment
#{deploy_dir}/etc/environment
, e.g./srv/app/etc/environment
#{configuration_dir}/environment
, e.g./etc/app/environment
#{runtime_dir}/runtime-environment
, e.g./var/run/app/runtime-environment
Later values override earlier values, so you can set defaults which get overridden in the deployment or runtime environment.
Systemd and OS
limit_nofile
, Limit on open files, systemd LimitNOFILE, default 65535.
umask
, process umask, systemd UMask, default 0027
restart_sec
: time to wait between restarts, systemd RestartSec, default 5 sec.
service_type
: :simple | :exec | :notify | :forking
. Default :forking
.
In theory, "modern" applications are not supposed to fork. The Erlang VM runs
pretty well in "foreground" mode, but it is really expecting to run as a
standard Unix-style daemon. Systemd expects foregrounded apps to die when their
pipe closes, so this library runs forking mode by default. It sets pid_file
to runtime_directory/#{app_name}.pid
and sets the PIDFILE
env var to tell
Distillery where it is. See
https://elixirforum.com/t/systemd-cant-shutdown-my-foreground-app-cleanly/14581/2
To run in foreground mode, set service_type
to :simple
or :exec
. Note
that in simple
mode, systemd doesn't actually check if the app started
successfully, it just keeps going. If something depends on your app being up,
:exec
may be better.
restart_method
: :systemd_flag | :systemctl | :touch
. Default :systemctl
Set this to :systemd_flag
, and the library will generate an additional
unit file which watches for changes to a flag file and restarts the
main unit. This allows updates to be pushed to the target machine by an
unprivilieged user account which does not have permissions to restart
proccesses. touch
the file #{flags_dir}/restart.flag
and systemd will restart the unit.
Runtime configuration
For configuration, we normally use a combination of build time settings, deploy time settings, and runtime settings.
The configuration settings in config/prod.exs
are baked into the release. We
can then extend them with machine specific configuration stored in the
configuration dir /etc/#{ext_name}
which are read by the app on startup.
In on-premises deployments, we might generate the machine-specific configuration when setting up the app. In cloud and other dynamic environments, we may run from a read-only image, e.g. an Amazon AMI, which gets configured for the environment when it starts.
We can also read configuration settings at runtime, e.g. getting the database host and login from a configuration store like AWS Systems Manager Parameter Store or etcd.
We might also have things that change dynamically, e.g. the IP address of the machine.
Conform is a popular way of making a
machine-specific config file. Set conform
to true
, and the library will
set CONFORM_CONF_PATH
to /etc/#{ext_name}/#{app_name}.conf
. Conform has been
depreciated in favor of TOML, so
you should use that instead.
This library suports three ways to get runtime config:
ExecStartPre
scripts
These scripts run before the main ExecStart
script runs.
You can specify multiple scripts in the exec_start_pre
var. If the name
starts with a slash, it is run directly, otherwise it is expected to be in the
directory specified by scripts_dir
(normally bin
in the deploy dir).
These scripts should generally write config to either the systemd configuration_dir
(/etc
) for persistent config, and under runtime_dir
(/run
)
for data which systemd should clean up on restart.
Wrapper script
Instead of running the main ExecStart
script directly, run a shell script which
sets up the environment, then exec
the main script.
This is most useful for things that are truly dynamic and may change on restart. For example, if the app is in a cluster, we might get the IP address of the primary network interface to set the node name.
Set runtime_environment_wrap
to true
and set
runtime_environment_wrap_script
to the name of the script. Default is the
deploy-runtime-environment-wrap
script from mix_deploy
.
Systemd starts units in parallel when possible, but we may need to enforce ordering.
Set runtime_environment_service_after
to the names of systemd units that the
script depends on. For example, set it to cloud-init.target
if you are using cloud-init to get
runtime network information.
Systemd service
We can run our own service to collect runtime data and configure the system.
Set runtime_environment_service
to true
and this library will create
a service which runs the script specified by runtime_environment_service_script
and make it a runtime dependency of the main script.
Security
paranoia
: enable systemd security options, default false
.
NoNewPrivileges=yes
PrivateDevices=yes
PrivateTmp=yes
ProtectSystem=full
ProtectHome=yes
PrivateUsers=yes
ProtectKernelModules=yes
ProtectKernelTunables=yes
ProtectControlGroups=yes
MountAPIVFS=yes
│
chroot
: Enable systemd chroot, default false
.
Systemd RootDirectory
is set to current_dir
. You can also set systemd ReadWritePaths=, ReadOnlyPaths=,
InaccessiblePaths=
with the read_write_paths
, read_only_paths
and inaccessible_paths
vars.