Server — MariaDB — Systemd

Every service on February runs under systemd. Most of the time that means you interact with it via systemctl start, systemctl stop, systemctl status, and journalctl. MariaDB is no different, but it is worth understanding a bit more about how the service unit is structured, what systemd does for it during startup and shutdown, and how to harden the unit without breaking anything.

The MariaDB service unit

The unit file installed by the Ubuntu package lives at /lib/systemd/system/mariadb.service. Do not edit this file directly: package upgrades will overwrite it. The right way to customise systemd unit files is with a drop-in override, covered later in this article.

Inspect the unit:

systemctl cat mariadb.service

A few things worth noting in the output.

Type=notify — MariaDB uses systemd’s service notification protocol. When MariaDB has finished initialising and is ready to accept connections, it sends a notification to systemd. Systemd considers the service started only after receiving this notification, not simply after the process launches. This means systemctl start mariadb blocks until MariaDB is actually ready, which is the correct behaviour for a service other things depend on.

ExecStartPre — before the main process starts, systemd runs a pre-start check to verify the configuration is valid. If mysqld cannot parse 50-server.cnf, the pre-start check fails and the service does not start. This is why a syntax error in the config file prevents startup with a clear error in journalctl rather than a confusing runtime failure.

Restart=on-failure — if MariaDB crashes unexpectedly, systemd will attempt to restart it. It will not restart on a clean shutdown, which is the right behaviour: if you deliberately stop the service, it stays stopped.

PrivateTmp=true — already present in the default unit. MariaDB’s temporary files are isolated in a private namespace rather than sharing the system /tmp.

LimitNOFILE — the open file descriptor limit is raised significantly above the default. MariaDB needs to hold open file handles for every table file and every active connection. The system default of 1024 is far too low.

Startup and shutdown behaviour

When you run sudo systemctl start mariadb, the sequence is:

1. Systemd runs the ExecStartPre check to validate the configuration.
2. Systemd starts the mysqld process.
3. If this is a restart after an unclean shutdown, InnoDB runs crash recovery before accepting connections.
4. MariaDB sends the READY=1 notification to systemd via the notification socket.
5. Systemd marks the service as active and systemctl start returns.

Step 3 is why MariaDB can take longer to start after a power loss than after a clean shutdown. Crash recovery is doing real work before the notification is sent.

When you run sudo systemctl stop mariadb, systemd sends SIGTERM to the main process. MariaDB handles this as a clean shutdown: it flushes all dirty pages to disk, writes a final checkpoint to the redo log, and exits. The binary log is also flushed cleanly. This is the shutdown state you want: InnoDB starts cleanly next time with no recovery needed.

If MariaDB does not shut down within the timeout defined in TimeoutStopSec, systemd sends SIGKILL. On a lightly loaded server this should never be necessary. If it is happening, something is blocking the shutdown, which warrants investigation.

Check how long the last start and stop took:

systemctl status mariadb | grep -E "Active:|since"
journalctl -u mariadb -n 30 | grep -E "started|stopped|ready"

The current security posture

Systemd provides a tool for auditing a service’s security configuration:

systemd-analyze security mariadb.service

This produces a table of security properties and an overall exposure score on a scale of 0 to 10, where lower is more secure. The default MariaDB unit on Ubuntu will score somewhere around 7 to 8, which is rated MEDIUM or UNSAFE. The default unit does not apply many of the available sandboxing directives.

The exposure score is a useful guide, not an absolute measure. A lower score means fewer kernel interfaces and filesystem paths are accessible to a compromised MariaDB process. It does not mean the service is invulnerable.

Adding a hardening override

Create a drop-in directory and override file:

sudo mkdir -p /etc/systemd/system/mariadb.service.d/
sudo nano /etc/systemd/system/mariadb.service.d/hardening.conf

Add the following content:

[Service]
# Filesystem restrictions
ProtectSystem=full
ProtectHome=true
ReadWritePaths=/var/lib/mysql /var/log/mysql /run/mysqld /tmp

# Process restrictions
NoNewPrivileges=true
PrivateDevices=true

# Kernel protections
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectControlGroups=true

# Misc
RestrictSUIDSGID=true
LockPersonality=true

A few notes on the choices made here.

ProtectSystem=full rather than strict: strict makes the entire filesystem read-only except for paths explicitly listed in ReadWritePaths. full makes /usr, /boot, and /etc read-only but leaves other paths accessible. The reason for full over strict here is that MariaDB’s plugin and socket paths are spread across several locations that are easier to handle with full than to enumerate exhaustively. If you want to apply strict, the additional paths needed are /var/lib/mysql, /var/log/mysql, /run/mysqld, /tmp, and /etc/mysql.

ProtectHome=true prevents MariaDB from reading anything under /home, /root, or /run/user. There is no reason a database process needs access to home directories.

ReadWritePaths explicitly grants write access to the paths MariaDB actually needs: the data directory, the log directory, the runtime socket directory, and /tmp for temporary tables.

NoNewPrivileges=true prevents the MariaDB process or any child process from gaining privileges beyond those it started with, even if a binary with the setuid bit is executed. This closes off a class of privilege escalation attacks.

PrivateDevices=true replaces /dev with a minimal set of safe pseudo-devices. MariaDB does not need access to raw hardware devices.

ProtectKernelTunables=true and ProtectKernelModules=true prevent the process from modifying kernel parameters via /proc/sys or loading kernel modules. A database process has no business doing either.

Applying and verifying

Reload systemd and restart MariaDB:

sudo systemctl daemon-reload
sudo systemctl restart mariadb

Confirm MariaDB started cleanly:

sudo systemctl status mariadb
sudo tail -n 20 /var/log/mysql/error.log

The error log should show a clean start with no warnings about filesystem access or permission errors. If you see errors about inaccessible paths, add those paths to ReadWritePaths in the override file and restart again.

Run the security analysis again to see the improvement:

systemd-analyze security mariadb.service

The score should improve noticeably from the baseline. The remaining exposure will be from properties that are either difficult to apply without breaking MariaDB (such as PrivateUsers, which conflicts with how MariaDB manages its own socket permissions) or from properties that add complexity without meaningful benefit for February’s setup.

Verify the full application stack is still working after the hardening:

dig february.home.arpa @127.0.0.1
sudo mariadb -e "SHOW STATUS LIKE 'Uptime';"
sudo journalctl -u powerdns-admin -n 10

All three should produce normal output. If PowerDNS or PowerDNS Admin are failing to connect to MariaDB after the hardening, the most likely cause is a socket path or runtime directory not listed in ReadWritePaths. Check journalctl -u pdns and journalctl -u powerdns-admin for the specific error.

Resource limits

The MariaDB unit already sets LimitNOFILE to a high value. For February’s modest workload, the defaults are appropriate. If the error log shows Too many open files errors as more services are added, increase the limit in the override:

[Service]
LimitNOFILE=32768

Similarly, if memory pressure becomes a concern as the server accumulates more running services, systemd’s MemoryMax directive can cap MariaDB’s memory usage:

[Service]
MemoryMax=512M

This should be set to at least the sum of innodb_buffer_pool_size, aria_pagecache_buffer_size, and the process overhead, with room to spare. Setting it too low will cause MariaDB to be killed by the OOM killer when it exceeds the limit, which is worse than not having the limit at all.

Keeping overrides across upgrades

Drop-in files in /etc/systemd/system/mariadb.service.d/ survive package upgrades. The base unit file in /lib/systemd/system/ may be updated by the package manager, but your overrides merge with it rather than being replaced. After a MariaDB upgrade, run systemctl daemon-reload and systemctl restart mariadb, then re-verify the hardening still works as expected. Occasionally a major upgrade changes something in the base unit that conflicts with an override, and the service fails to start until the override is adjusted.