Automated Security Patching & Maintenance Windows with unattended-upgrades and needrestart

4 min read

Maintaining security on a fleet of home servers and cloud VPS instances requires continuous, timely application of critical kernel and package CVE patches. Manually running apt update && apt upgrade across multiple hosts inevitably leads to delayed updates and vulnerable attack surfaces. This guide establishes a production-grade automated patching architecture using unattended-upgrades and needrestart on Ubuntu 24.04 LTS and Debian 12, complete with scheduled reboot windows and instant alert notifications.

⚡ Quick Runbook: 60-Second Implementation

Deploy the baseline configuration in a single root execution block:

# 1. Install required packages
sudo apt-get update && sudo apt-get install -y unattended-upgrades needrestart bsd-mailx

# 2. Enable unattended-upgrades
sudo dpkg-reconfigure -plow unattended-upgrades

# 3. Configure automated 04:00 AM reboot maintenance window
sudo tee /etc/apt/apt.conf.d/52homelab-maintenance <<'CONFIG'
Unattended-Upgrade::Allowed-Origins {
    "${distro_id}:${distro_codename}-security";
    "${distro_id}ESMApps:${distro_codename}-apps-security";
    "${distro_id}ESM:${distro_codename}-infra-security";
};
Unattended-Upgrade::Package-Blacklist {
    // Hold critical packages from auto-upgrading without manual testing
    // "docker-ce";
    // "zfs-dkms";
};
Unattended-Upgrade::AutoFixInterruptedDpkg "true";
Unattended-Upgrade::MinimalSteps "true";
Unattended-Upgrade::InstallOnShutdown "false";
Unattended-Upgrade::Remove-Unused-Kernel-Packages "true";
Unattended-Upgrade::Remove-New-Unused-Dependencies "true";
Unattended-Upgrade::Remove-Unused-Dependencies "true";
Unattended-Upgrade::Automatic-Reboot "true";
Unattended-Upgrade::Automatic-Reboot-WithUsers "true";
Unattended-Upgrade::Automatic-Reboot-Time "04:00";
CONFIG

# 4. Configure needrestart for automated daemon restarts
sudo sed -i "s/#\$nrconf{restart} = 'i';/\$nrconf{restart} = 'a';/" /etc/needrestart/needrestart.conf

# 5. Test configuration in dry-run debug mode
sudo unattended-upgrade --dry-run --debug

Architecture & Operational Flow

An automated maintenance pipeline balances security velocity against system uptime. The workflow operates as follows:

  • APT Daily Timer: Systemd invokes apt-daily.timer to synchronize repository metadata.
  • CVE Identification: apt-daily-upgrade.timer parses installed packages against active security feeds (e.g. noble-security). Only CVE security updates are installed; major version bumps and standard feature releases are deferred to scheduled administrative maintenance.
  • Kernel Cleanup: Old, inactive kernel images are automatically purged to prevent /boot partition exhaustion.
  • Service Recycling (needrestart): When shared libraries (such as glibc or openssl) are updated, needrestart inspects running processes and recycles the affected systemd daemons automatically without requiring a full machine reboot.
  • Scheduled Reboot Window: If a new Linux kernel is deployed, the system registers a reboot flag (/var/run/reboot-required). Rather than restarting immediately in the middle of active operations, the reboot is executed precisely at 04:00 AM local time.

Step 1: Fine-Tuning Package Blacklists

Certain infrastructure components—such as OpenZFS kernel modules (zfs-dkms) or container runtimes (docker-ce)—should be tested manually before upgrading on production storage servers or hypervisors. Add these to the package blacklist in /etc/apt/apt.conf.d/50unattended-upgrades:

Unattended-Upgrade::Package-Blacklist {
    "docker-ce";
    "docker-ce-cli";
    "containerd.io";
    "zfs-dkms";
    "zfsutils-linux";
    "nvidia-driver-.*";
};

Step 2: Automated Failure & Reboot Alerting

To receive instant webhook notifications on Discord, Slack, or Ntfy whenever an automated reboot occurs, deploy a simple systemd one-shot service that fires on boot if a restart was scheduled:

sudo tee /etc/systemd/system/notify-boot.service <<'CONFIG'
[Unit]
Description=Notify Administrator on System Boot
After=network-online.target
Wants=network-online.target

[Service]
Type=oneshot
ExecStart=/usr/bin/curl -s -d "Server $(hostname) has rebooted successfully following scheduled maintenance." https://ntfy.example.com/homelab-alerts

[Install]
WantedBy=multi-user.target
CONFIG

sudo systemctl daemon-reload
sudo systemctl enable notify-boot.service

Step 3: Verification & Log Inspection

Verify that your unattended-upgrades configuration executes cleanly without syntax errors:

# Execute dry run with full verbosity
sudo unattended-upgrade --dry-run --debug

# Review past upgrade logs
cat /var/log/unattended-upgrades/unattended-upgrades.log

# Check if a reboot is currently pending on the server
if [ -f /var/run/reboot-required ]; then
    echo "Reboot is required by: $(cat /var/run/reboot-required.pkgs)"
else
    echo "No reboot required. System is running the latest kernel."
fi

Related Runbooks in This Series

Tags:

← Previous:
Next: →