Background queue (Redis-optional)

The background queue is the async backbone: OTP/welcome emails, the whole transactional catalogue, push fan-out, and job-match notifications all run through it. Its defining property is that Redis is optional.

The contract

One abstract QueuePort that producers depend on:

abstract class QueuePort {
  enqueue<T>(queue, name, payload, opts?): Promise<void>;
  health(): Promise<{ driver: 'bullmq'|'database'; depth; dlqDepth; healthy }>;
}

Consumers don't poll — they register a handler with a QueueProcessorRegistry (a Map keyed by queue:name) in their onModuleInit. The active adapter's worker resolves and runs the handler when a job comes due. Producers and consumers never know which driver is live.

Boot-time selection

A factory picks exactly one adapter at startup, driven by QUEUE_DRIVER:

database          → DbQueueAdapter (explicit)
bullmq            → BullMqQueueAdapter (explicit; surfaces loudly if Redis is down)
auto (default):
   no Redis config            → DbQueueAdapter        (log: "no Redis; DB queue")
   Redis config + ping OK     → BullMqQueueAdapter
   Redis config + ping fails  → DbQueueAdapter        (WARN: "Redis unreachable; falling back")

The ping is a short-timeout ioredis connect that never throws — so a misconfigured or down Redis degrades to the DB queue with a warning instead of crashing the boot. This is the deliberate override of the old "fail-fast" rule (ADR-0020).

The two adapters

BullMqQueueAdapter — one BullMQ Queue + a managed Worker per logical queue, attempts:3 with exponential backoff, and a dedicated dead-letter queue (<queue>-dlq; note BullMQ forbids : in names) for exhausted jobs. Graceful close on shutdown.

DbQueueAdapter + DbQueueWorker — the durable fallback, backed by a Prisma QueueJob table. Every pollIntervalMs the worker:

1. Recovers stale leases — any active job whose lease is older than the visibility timeout is reset to pending. This is the self-healing that fixes the legacy outage where jobs got stuck until a manual queue:restart.
2. Claims due jobs atomically — SELECT … FOR UPDATE SKIP LOCKED inside a transaction, flipping them to active with a lockedAt/lockedBy lease, so multiple workers never grab the same job.
3. Runs each handler — success → completed; failure → attempts++ and either re-schedule with exponential backoff (pending, runAt = now + backoff·2^n) or, when exhausted, route to the dead-letter state (dead).

Both implement OnModuleDestroy for a graceful drain (stop accepting work, let in-flight jobs finish).

Health probe

GET /api/health/queue returns { driver, depth, dlqDepth, healthy } wrapped in the standard envelope — so you can see which path is live and whether the queue is backing up.

Why this shape

Because the choice is a port with two adapters, the rest of the app is written once and works either way; tests run on the DB path with zero infrastructure; and switching a deployment to Redis is a single env var. The cost is implementing a correct DB queue (locking, backoff, DLQ, lease recovery) — which is exactly what the unit tests pin.

Gotchas

• e2e is pinned to QUEUE_DRIVER=database in global-setup so the suite needs no Redis.
• The DB worker polls (default 1s), so DB-path delivery has up to ~1s latency; BullMQ is near-instant. Both are durable across restarts.