DB-유

Unattended Security Updates on Ubuntu

· linux, ubuntu, security, apt

Patching isn't hard. Remembering to apt-get upgrade on a dozen machines is, and I skip it whenever I'm busy, which is always. Ubuntu's unattended-upgrades does it for me: install security updates, reboot at a quiet hour if needed, mail a summary.

This is Ubuntu 18.04 (16.04 has the same files), limited to the security pocket, with controlled reboots and a blacklist for packages that need a human. Reports need some way for the server to send mail, like the mailx SMTP setup or a local Postfix relay.

Turning it on

Most Ubuntu server images already have it:

sudo apt-get install -y unattended-upgrades apt-listchanges
sudo dpkg-reconfigure -plow unattended-upgrades

Choose "Yes" at the prompt. This writes /etc/apt/apt.conf.d/20auto-upgrades:

APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Unattended-Upgrade "1";

Values are in days: refresh lists daily, run the upgrader daily. On 18.04 two systemd timers drive it, apt-daily.timer (downloads) and apt-daily-upgrade.timer (installs), not cron.

What to upgrade

The main config is /etc/apt/apt.conf.d/50unattended-upgrades. The default Allowed-Origins already limits it to security updates; I check that only that line and ESM (if applicable) are active and the rest are commented out:

Unattended-Upgrade::Allowed-Origins {
        "${distro_id}:${distro_codename}-security";
        "${distro_id}ESM:${distro_codename}";
//      "${distro_id}:${distro_codename}-updates";
//      "${distro_id}:${distro_codename}-proposed";
//      "${distro_id}:${distro_codename}-backports";
};

Enabling -updates adds bug-fix releases too. I leave it on for general-purpose web servers and off for database hosts, which I upgrade deliberately.

Packages that need a human

Some things shouldn't upgrade unless someone is watching: the database server, a kernel on a host with out-of-tree modules, anything with a manual migration step.

Unattended-Upgrade::Package-Blacklist {
        "mysql-server";
        "mysql-server-5.7";
        "postgresql-10";
        "linux-image-";
        "docker-ce";
};

Entries are regexes anchored at the start, so linux-image- matches every kernel package. Blacklisting the kernel is a trade-off: its security fixes wait for my maintenance window. On hosts where a surprise reboot is unacceptable, I'll take that.

Reboots and cleanup

Further down the same file:

Unattended-Upgrade::Remove-Unused-Kernel-Packages "true";
Unattended-Upgrade::Remove-Unused-Dependencies "true";
Unattended-Upgrade::Automatic-Reboot "true";
Unattended-Upgrade::Automatic-Reboot-WithUsers "false";
Unattended-Upgrade::Automatic-Reboot-Time "04:30";

Automatic-Reboot only reboots when /var/run/reboot-required exists, which a kernel or libc update creates. WithUsers "false" skips the reboot if anyone is logged in, so a late-night troubleshooting session doesn't get cut off. With the kernel blacklisted, reboots almost never trigger anyway.

Mail reports

Unattended-Upgrade::Mail "ops@example.com";
Unattended-Upgrade::MailOnlyOnError "true";

MailOnlyOnError keeps the inbox quiet; you hear about it only when a package fails to configure. I set it to false for the first couple of weeks to see what's being applied. Reports go out via the local mail command, so mailutils, bsd-mailx or s-nail must be installed and set up to relay.

Timer schedule

The default timers fire at random times to spread load on the mirrors. I want a predictable window across a fleet, so I override the upgrade timer:

sudo systemctl edit apt-daily-upgrade.timer

[Timer]
OnCalendar=
OnCalendar=*-*-* 04:00
RandomizedDelaySec=20m

The empty OnCalendar= clears the packaged value before setting yours. Reload with sudo systemctl daemon-reload and check systemctl list-timers apt-daily*.

Dry run

This shows what would happen without changing anything:

sudo unattended-upgrade --dry-run --debug

It lists the allowed origins, every candidate package, and whether it's blacklisted. After the first real run, read the logs:

sudo tail -n 50 /var/log/unattended-upgrades/unattended-upgrades.log
sudo tail /var/log/unattended-upgrades/unattended-upgrades-dpkg.log

systemctl status apt-daily-upgrade.timer shows the next scheduled trigger.

Notes

  • Allowed origins are: ... then No packages found that can be upgraded unattended, while apt list --upgradable shows security updates: origin strings don't match. Compare apt-cache policy o= and a= fields with what you wrote; on 18.04 it must be Ubuntu:bionic-security.
  • The upgrader never runs and systemctl list-timers shows the timer inactive: APT::Periodic::Unattended-Upgrade is 0. Rerun dpkg-reconfigure or edit 20auto-upgrades.
  • E: Could not get lock /var/lib/dpkg/lock-frontend: apt-daily was running when you ran apt-get by hand. Wait a minute.
  • No mail arrives: /usr/bin/mail is missing. apt-get install bsd-mailx and set up relay; Unable to send mail in the log confirms it.
  • A service restarted during the update window and nobody expected it: needrestart (on 18.04) restarting services with updated libraries. Set $nrconf{restart} = 'l'; in /etc/needrestart/needrestart.conf to list instead.

It's no substitute for a real patch schedule, but it shrinks the gap between "patch released" and "someone got round to it" from weeks to a day.