Half the systems we’ve audited in the last two years have a queue table in Postgres. A jobs table with a status column, a claimed_at timestamp, and a polling worker. It’s the cheapest queue you can build, and FOR UPDATE SKIP LOCKED makes it perform much better than people expect.
The question we get asked most often is whether to migrate to a real broker. The answer is almost always “not yet.”
Why Postgres is fine
For most workloads, a single Postgres queue handles thousands of jobs per minute without breaking a sweat. The pattern is simple:
- A worker runs
SELECT id FROM jobs WHERE status = 'pending' ORDER BY priority, created_at LIMIT 1 FOR UPDATE SKIP LOCKED - It updates the row to
status = 'running'withclaimed_at = now() - It does the work
- On success it deletes the row (or moves to a history table); on failure it bumps a retry counter
SKIP LOCKED is the magic. Without it, every worker would block on the same row. With it, each worker grabs the next available job atomically. Postgres has supported it since 9.5.
The operational benefits are huge: you already monitor Postgres, you already back it up, your jobs participate in the same transactions as your business data, and you can query the queue with SQL. Failed jobs become a normal database query, not a webhook to a vendor dashboard.
When it breaks
Three symptoms tell you it’s time to leave.
First, table bloat. A queue table that ingests millions of rows per day will accumulate dead tuples even with autovacuum. Once VACUUM can’t keep up, query latency starts oscillating. Partitioning by date helps; eventually it doesn’t.
Second, the worker fleet starts dominating database connections. Postgres connections are expensive. If you’re running 50 workers each holding a long-lived connection, you’re spending half your DB capacity on the queue. PgBouncer helps. So does reducing worker count by making each worker faster.
Third, you need real fan-out — one job triggers ten downstream jobs across multiple services. Postgres can do this with LISTEN/NOTIFY, but the model gets fragile. This is where a real broker (NATS, Kafka, RabbitMQ, even SQS + SNS) earns its keep.
The migration path
We’ve done this migration three times. The pattern that works:
Start by isolating queue access behind an interface. The application calls Queue.enqueue(job) and Worker.handle(callback). The implementation is Postgres for now. This refactor is half the migration.
When you’re ready to switch, dual-write for two weeks. Every enqueue goes to both Postgres and the new broker. Workers consume from the new broker. The Postgres queue acts as a recovery rail if anything goes wrong. After two weeks, drop the Postgres write.
Don’t skip the dual-write phase. Every team that has skipped it has lost jobs.
Sizing the decision
The rough rule we use: under 100k jobs per day, stay on Postgres unless you have a specific need. Between 100k and 1M, evaluate based on workload shape (steady is fine; burst is harder). Above 1M, you’ve usually outgrown it, and the cost of a broker is the smaller line item compared to the operational headache.
One caveat: if your jobs need delivery guarantees stronger than at-least-once with idempotent workers, you have a different problem, and Postgres isn’t the bottleneck. Solve idempotency first.