Automatic security updates
Most compromised servers are not broken into cleverly. They are running something with a published fix that nobody applied.
Debian and Ubuntu
apt install unattended-upgrades apt-listchanges
dpkg-reconfigure -plow unattended-upgrades
By default it applies security updates only, which is the right setting. Check what it would do without doing it:
unattended-upgrades --dry-run -d
And confirm afterwards that it is actually running — the single most common failure is a configuration that was installed and never took effect:
grep -c " upgrade" /var/log/unattended-upgrades/unattended-upgrades.log
RHEL, Rocky and AlmaLinux
dnf install dnf-automatic, set
upgrade_type = security and apply_updates = yes in
/etc/dnf/automatic.conf, then
systemctl enable --now dnf-automatic.timer.
What to automate and what not to
- Automate
- Security updates to libraries and daemons. OpenSSL, the web server, the SSH daemon, the kernel package itself. These are the ones that are actively exploited.
- Do not automate
- Major version upgrades, database engines, and your control panel. A database that restarts into a new major version in the middle of the night is its own outage.
- Decide deliberately
- Reboots. A kernel update does nothing until the machine restarts.
Exclude the packages you want to control by hand with
Unattended-Upgrade::Package-Blacklist, and pin them so a manual
apt upgrade does not catch them either.
The reboot question
You can let it reboot for you:
Unattended-Upgrade::Automatic-Reboot "true";
Unattended-Upgrade::Automatic-Reboot-Time "04:00";
Sensible for a stateless machine behind a load balancer. Not sensible for the only database server you have. If you leave it off, watch for pending reboots instead:
[ -f /var/run/reboot-required ] && echo "reboot required"
Put that in your monitoring. A machine that has needed a reboot for four months is running a kernel with known holes in it. See kernel upgrades for live patching, which buys time but is not a substitute.
Things that bite
- Configuration file prompts. An unattended run that hits a
conffile question stalls. Set
Dpkg::Options { "--force-confdef"; "--force-confold"; }so your existing configuration is kept. - A full /boot. Old kernels accumulate until updates fail.
apt autoremove --purgeregularly, and alert on/bootusage. - Services that need a restart. Updating a library does not
restart what links against it.
needrestarttells you what is still running old code. - Silence is not success. Have it mail you, or check the log in monitoring. A broken updater looks exactly like a machine with no updates available.
Still stuck? Mail support@novogara.com — an engineer answers, at any hour. Back to the knowledge base