The healthcheck that DoSed itself — a throttled /healthz

After reviving notes, it served users fine but reported unhealthy forever. The container was running, the app logs were clean, https://notes/ returned 200 — yet docker inspect kept saying Health=unhealthy. This is why, and why it's left open rather than silently patched.

The symptom

notes-app-1   Up (unhealthy)          # running, serving, but health flips to unhealthy
# in-container probe:
GET /healthz → 429 {"statusCode":429,"message":"ThrottlerException: Too Many Requests"}

The healthcheck returns 429, not a 500 or a timeout. The app is alive — it's being rate-limited by its own rules.

Root cause: a global throttler + a health probe that hammers it

The app registers the NestJS throttler globally:

ThrottlerModule.forRoot([{ ttl: 15 * 60_000, limit: 10 }]),   // 10 requests / 15 min
providers: [ { provide: APP_GUARD, useClass: ThrottlerGuard } ] // applies to EVERY route

And the Docker healthcheck knocks every 15 seconds:

healthcheck:
  test: ["CMD","node","-e","fetch('http://127.0.0.1:3100/healthz')..."]
  interval: 15s

/healthz is an ordinary Nest route, so it's under the global throttler. At one call per 15s that's ~4/min ≈ 60 requests per 15-minute window from 127.0.0.1 — against a limit of 10. So the probe exhausts its own quota in about 2.5 minutes, after which /healthz returns 429, the healthcheck fails, and the container flips to unhealthy — and stays there. It looked healthy right after boot (the first ~10 probes pass) and then flipped, which is exactly this window filling up.

The same 10-per-15-min limit applies to real visitors (per source IP), so a person browsing a handful of notes could hit 429 too. (Static assets served via Express useStaticAssets bypass the Nest guard, so only Nest routes are counted — but that still includes the SPA shell, the API, and the health route.)

The trap: a healthcheck must probe liveness without changing state or consuming budget. Pointing it at a rate-limited endpoint turns the watchdog into a self-inflicted denial of service — the thing meant to prove the service is up is the thing marking it down.

Status: diagnosed, not cured (on purpose)

The fix is small but it is a change to the app's own behaviour/security posture, not host recovery — so it belongs to the owner, not a wake-up sweep:

@SkipThrottle() on /healthz — health/liveness endpoints should never be throttled. This alone stops the self-DoS and lets the container report healthy.
Reconsider the global limit — 10 / 15min across the whole app is unusually tight (the throttler's own example is 10 / 1min). Either loosen it, or scope the guard to the auth/login routes that actually need brute-force protection instead of registering it as a blanket APP_GUARD.

Until the owner decides, notes is honestly recorded as up but rate-limited, not "healthy" — see the e2e sweep, which classifies a 429 as app responding, throttled rather than a pass.

Generalise it

Never aim a healthcheck at a throttled, authenticated, or side-effecting route — give it a dedicated, unthrottled liveness path. And be wary of a global APP_GUARD throttler: it silently catches health probes, readiness checks, and asset-ish routes you never meant to limit. If a container is running + serving but perpetually unhealthy, suspect the healthcheck endpoint's own middleware before the app.