Job queue

A list of slow tasks your app hands off to run later in the background, so the user gets a response now instead of waiting.

do the slow thing after the responsea to-do list my server works throughbackground jobstask queuedon't make the user wait for the email to sendqueing systemprocess this later in the backgroundthe request times out because the work takes 30 seconds

See it

Live demo coming soon

What it is

A job queue is a durable to-do list your app writes to and separate processes read from. The web request does the fast part (validate, save a row, enqueue a job) and returns immediately; the slow part (send the email, transcode the video, call the billing API, generate the PDF) happens after. Common stacks: BullMQ on Redis, Sidekiq in Ruby, Celery in Python, Postgres-backed queues like Graphile Worker or river, and hosted ones like SQS, Inngest, or Trigger.dev.

Reach for it whenever work is slow, flaky, or bursty: anything calling a third-party API, anything touching files, anything that could take longer than a request timeout. The queue also gives you back-pressure. A traffic spike makes the backlog grow rather than making the site fall over, and you absorb it by adding workers.

The rule people learn the hard way: exact delivery semantics vary by product and config, but design every durable job system as if it were at-least-once, because a worker can finish the work and then die before it acknowledges the job, and the queue has no way to know. So a job can run twice. Make handlers idempotent at the point of effect, keyed on something stable, or you will double-charge somebody. Two more: never enqueue a job inside a database transaction that might roll back, since the worker can pick it up before you commit and find no row. Enqueueing after the commit has its own hole (the process can die in the gap and the job is simply lost), so use it when losing the job is recoverable and write the job into an outbox table in the same transaction when it is not. And wire up a dead letter queue plus retries with backoff, because failed jobs that vanish silently are worse than errors on screen.

Ask AI for it

Move this slow work out of the request path and into a background job queue using BullMQ with Redis (or Graphile Worker if I want to stay on Postgres). Define a typed job payload carrying only ids, not whole objects, and return to the user right away. Do not enqueue from inside the transaction and do not rely on enqueueing right after it either: insert the job into an outbox table in the same transaction as the data, and have a dispatcher move unsent outbox rows onto the queue and mark them sent. Enforce idempotency at the effect boundary, not just at the queue: give each job a unique operation key and either insert that key with a unique constraint in the same transaction as the effect, or pass it as the provider's idempotency key (Stripe, the email API) so a retry after a crash cannot repeat the charge or the send. Configure exponential backoff, a max attempt count, and a dead letter queue for jobs that exhaust retries, and log job id, attempt number, and duration on every run.

You might have meant

workeridempotencycron jobdead letter queue