The Long Dark — when the whole box is paused and resumed

The box came back from being off and every clock disagreed. This is how to recognise a hypervisor pause/resume, why the numbers lie, and the audit that should follow every wake.

The symptom: clocks that contradict each other

Right after the box came back, snapshot and the raw tools told three different stories about how long it had been up:

uptime              → "up 7 minutes"
who -b / journal    → booted 2026-07-10 23:16   (now 03:25 → ~4h ago)
docker ps           → every container "Up 4 hours"
ps aux              → dockerd, init, xorg all "started 03:16" (7 min ago)

Seven minutes, or four hours? Both, and that's the tell.

Root cause: a frozen monotonic clock

The VM was paused (suspended) by the host for most of those four hours, then resumed ~7 minutes before I looked. Two different clocks explain the split:

CLOCK_MONOTONIC — counts un-paused running time. It freezes while the guest is paused. uptime, and the process "start" ages ps computes as now − age, are derived from it — so they only counted the 7 minutes since resume, and ps back-dated every long-lived process to "03:16".
CLOCK_REALTIME (wall clock, kept honest by the RTC/NTP) — kept advancing the whole time. who -b, the systemd journal timestamps, and Docker's StartedAt (which is a wall-clock instant) all use it — so they correctly show ~4 hours since boot.

The gap between the two — ~4h wall vs ~7min monotonic — is the pause duration. Nothing was actually wrong; the guest simply lost time it never felt.

The lesson: if uptime and docker ps/who -b disagree by a large, consistent margin, you weren't rebooted — you were paused and resumed. Don't chase a phantom reboot; audit for the side effects of the freeze instead.

The side effects worth checking

A pause doesn't corrupt anything, but a few things get confused by the frozen clock:

systemd start-timeouts that were mid-flight when the pause hit fire on resume. Here xrdp.service had a slow PID-file race at boot; the 90s start timeout froze during the pause and only "elapsed" on resume, so systemd terminated it and marked it (and xrdp-sesman) failed. A clean systemctl restart — with no pause to freeze the timer — brought both back to active. The failure was an artifact of the freeze, not a real fault.
Crashlooping containers. A container whose dependency wasn't ready can spin its restart counter across the wake. Check docker ps for Restarting / unhealthy first.

The post-resume audit (run this every wake)

snapshot                                   # vhosts, containers, certs, disk at a glance
docker ps                                  # any Restarting / (unhealthy) / Exited?
systemctl --failed                         # any unit that didn't survive the wake
# then drive every public host over its LIVE https URL, desktop + mobile,
# capturing console/page errors (see the e2e sweep in /root/.verify-resume/sweep.mjs)

On this particular wake the sweep found 15 of 16 vhosts perfectly healthy and one real fire, plus a handful of latent issues worth hardening. Each got its own page:

notes was down — its database couldn't read its own password file. See When a container can't read its own secret.
A renewed cert would never have been served — the certbot deploy hook was missing. See Renewed certs that never get served.
notes reports "unhealthy" forever — a healthcheck pointed at a rate-limited endpoint. See The healthcheck that DoSed itself (still open — owner's call).
Housekeeping: rc-local.service was failing on a stale swapon /home/swapfile (real swap is /swapfile via fstab) — struck the dead line and made its GUI restarts --no-block; and the nginx apex default_server was missing http2 while every other vhost had it, which produced "protocol options redefined" warnings — aligned it so nginx -t is clean.

Generalise it

"The server was off" usually means one of three things — a real reboot (check who -b moved), a crash (check journalctl -b -1 -p err), or, on a cheap cloud VM, a host-driven pause/resume with this exact clock signature. The recovery is the same either way: don't trust memory for what's running — query the system, walk every service to a real 200, and clear any unit that --failed during the confusion. See also Why tmux kept dying — OOM resilience for the memory-pressure cousin of "it was fine, then it wasn't."