Engineering notes on reliable job delivery.
Practical, opinionated writing about job queues, LLM reliability, retries, rate limits, and running async work in production.
Rate limiting strategies for API-dependent backends
Per-worker limits multiply into storms. Here's how fixed-window, token-bucket, and sliding-window strategies actually behave under load — and how one shared budget keeps a whole fleet under an upstream's ceiling.
Idempotency keys: making job publishing safe to retry
A network blip between you and your queue can leave you unsure whether a job got enqueued. An idempotency key turns that ambiguous retry into a no-op — here's how publish-boundary dedup works and why your worker still needs to be idempotent.
Backpressure: handling 429, 503, and 529 without losing work
A rate-limited request didn't fail — it was told "not now." Treating backpressure as a failed attempt burns your retry budget and drops work. Here's the defer model that fixes it.
Why job retries matter (and how to get them right)
If your background jobs don't have backoff, jitter, idempotency, and a dead-letter queue, you are silently dropping work — and never finding out which jobs you lost.
Running jobs longer than a webhook timeout: ack mode
A synchronous webhook has a hard ~15-second ceiling, but LLM generations and slow third-party calls run longer. Ack mode lets you return 200 fast and report the real outcome out of band.
Dead-letter queues: catching and replaying failed jobs
When a job exhausts its retries, it should land somewhere you can inspect — not vanish into a log file. A guide to DLQs, per-attempt audit trails, and the inspect-fix-replay loop.
How to verify webhook signatures (HMAC-SHA256)
Your webhook endpoint is a public URL anyone can POST to. Here's how to prove a request actually came from your queue — HMAC-SHA256 over the raw body, constant-time comparison, and per-queue secrets.
Why every AI app needs a reliable job queue
LLM calls fail, hit rate limits, and time out — and inline calls are quietly the biggest reliability risk in modern AI apps. Here's how to fix it.
Queue vs workflow engine: what startups actually need
Most teams reach for a workflow engine before they need one. Here's when a managed queue is the right first move — and the specific signals that mean you've actually outgrown it.
How to handle OpenAI rate limits in production
Tokens-per-minute limits, retry-after headers, and shared buckets across your fleet — a practical guide that doesn't end in 429s.
Building async infrastructure without overengineering
When to reach for Redis, when to reach for a hosted queue, and when to skip both. A guide to staying simple as your traffic grows.