Kernel Updates and Live Patching Options on Linux

Kernel updates are the one category of patch that traditionally needs a reboot, and on a database server a reboot means a maintenance window, a failover, or an argument with whoever owns the application. That tension is why live patching exists. I'll cover the boring part first, applying kernel updates on CentOS 7/8 and Ubuntu 18.04 and knowing when a reboot is actually pending. Then I'll get to what live patching realistically offers.

1. Running vs. installed

uname -r
rpm -q kernel                         # CentOS
dpkg -l 'linux-image-*' | grep ^ii    # Ubuntu

On CentOS the two lists often differ: rpm -q kernel shows three installed versions and uname -r shows an older one, meaning updates were applied but the machine has not rebooted since. That is the state to watch for.

2. Applying the update

CentOS 7 and 8:

sudo yum update kernel

Kernels are "installonly" packages, so the new one is installed alongside the old ones rather than replacing them, and installonly_limit=3 in yum.conf/dnf.conf trims the oldest. The new kernel becomes the default GRUB entry. Confirm with:

sudo grubby --default-kernel
sudo grubby --info=ALL | grep ^kernel

Ubuntu 18.04:

sudo apt update
sudo apt install --only-upgrade linux-image-generic linux-headers-generic

or plain apt upgrade, which includes the kernel metapackages. Old kernels accumulate under /boot, and on a small /boot partition this eventually breaks updates; apt autoremove cleans them up.

3. Is a reboot pending?

Ubuntu leaves a marker file:

cat /var/run/reboot-required
cat /var/run/reboot-required.pkgs

CentOS has no such file. Use needs-restarting from yum-utils:

sudo needs-restarting -r

It compares the running kernel and core libraries against the installed versions and exits 1 if a reboot is needed. A simpler check is comparing uname -r with the newest installed kernel:

[ "$(uname -r)" = "$(rpm -q --last kernel | head -1 | awk '{print $1}' | sed 's/kernel-//')" ] && echo current || echo reboot-needed

I run one of these from the monitoring system so "kernel updated, not rebooted" shows up as a warning rather than being discovered months later.

4. Falling back to the old kernel

On CentOS, the previous kernel is still in the GRUB menu, and you can make it the default without rebooting first:

sudo grubby --set-default /boot/vmlinuz-3.10.0-1062.4.1.el7.x86_64

On CentOS 8 with BLS (boot loader spec) entries, grubby still works and grub2-set-default is the alternative. On Ubuntu, hold Shift at boot to show the GRUB menu, or set GRUB_DEFAULT in /etc/default/grub to the exact menu entry and run update-grub.

5. Live patching

Live patching applies fixes for selected security vulnerabilities to the running kernel without rebooting. Three things to understand before relying on it: it covers only patches that can be expressed as function replacements, typically CVE fixes, not whole kernel upgrades; it does not remove the need to reboot eventually; and on every platform it is a paid or account-gated service.

Ubuntu: Canonical Livepatch. Free for up to three machines with an Ubuntu One account, paid via Ubuntu Advantage beyond that. It works on the GA kernel of LTS releases:

sudo snap install canonical-livepatch
sudo canonical-livepatch enable <your-token>
sudo canonical-livepatch status --verbose

The token comes from the Livepatch page of your Ubuntu One account. Status shows the running kernel, which CVEs have been patched live, and whether a reboot is still advised for anything it could not cover.

RHEL: kpatch. Red Hat ships kpatch and kpatch-patch packages for RHEL 7.7+ and 8 as part of the subscription. On CentOS the kpatch tooling exists in the repos but Red Hat does not publish the actual patch modules for CentOS kernels, so in practice there is nothing to apply. Do not expect yum install kpatch on CentOS to give you live patching.

Commercial: KernelCare. CloudLinux sells KernelCare for CentOS, Ubuntu, Debian and others. It is a per-server subscription, installs as a small agent, and checks for patches every few hours. It is the realistic option for a CentOS fleet that cannot reboot on Red Hat's schedule.

For most of my servers, the honest answer is that a well-run reboot schedule plus HA at the database or application layer is cheaper than any of the above. Live patching earns its keep on single machines with no failover where a critical CVE lands mid-quarter.

After the reboot

Check:

uname -r
sudo journalctl -b -p err
sudo dmesg | grep -iE 'error|fail' | head

The running version should match the newest installed package, and the boot should show no new errors. For Livepatch, canonical-livepatch status should read patchState: applied and list the CVE numbers.

Things that went wrong for me

• CentOS boots the old kernel after an update: the default entry was not updated because /boot was full when the initramfs was built. df -h /boot, remove old kernels with package-cleanup --oldkernels --count=2, then yum reinstall kernel-<version>.
• grubby fatal error: unable to find a suitable template: the /boot/grub2/grub.cfg is missing or the BLS directory is empty. Regenerate with grub2-mkconfig -o /boot/grub2/grub.cfg.
• canonical-livepatch enable fails with Kernel ... is not supported: HWE kernels and non-LTS releases are outside Livepatch coverage; switch to the GA kernel (linux-image-generic).
• A third-party module (ZFS, VirtualBox, VMware Tools) stops working after the kernel update: it must be rebuilt. DKMS handles this automatically for modules registered with it; check dkms status.

Monitor the running-vs-installed gap and patch on a fixed cadence. Live patching is a bridge to the next window, not a replacement for it.