TL;DR: SKIP LOCKED is a PostgreSQL SELECT ... FOR UPDATE option that makes it easy to build efficient worker queues: rows locked by other transactions are skipped rather than blocked. That lets multiple workers concurrently fetch work without stepping on each other’s toes, avoiding hot-row contention and simplifying retry logic.
The problem: multiple workers picking up tasks
Imagine a table jobs that stores work to be processed. If many workers attempt to claim the same ready job at once, you get contention:
- naive
SELECTthenUPDATEapproaches can lead to races where multiple workers try to process the same job (lost updates or duplicated work) - using
SELECT ... FOR UPDATEwithoutSKIP LOCKEDcauses workers to block on locked rows and reduces throughput.
We want a simple, safe pattern where multiple workers can concurrently claim available jobs without blocking each other and without duplicating work.
SKIP LOCKED: what it does
SKIP LOCKED is used with SELECT ... FOR UPDATE (or FOR NO KEY UPDATE) and tells PostgreSQL to skip rows that are already locked by other transactions instead of waiting for the lock to be released.
Behavioral points:
- Rows that are locked by other transactions are omitted from the result set.
- Because you do
FOR UPDATE, rows returned are locked for your transaction and safe to update/consume. - It is a non-blocking way to claim rows — perfect for worker queues.
Important caveat: SKIP LOCKED provides no ordering guarantees relative to skipped rows — you’re accepting “best-effort” fairness. That’s fine for many queue scenarios but not for strict ordering requirements.
Task queue pattern using SKIP LOCKED (SQL)
Schema (simple example):
CREATE TABLE jobs (
id BIGSERIAL PRIMARY KEY,
payload jsonb NOT NULL,
state text NOT NULL DEFAULT 'queued',
locked_at timestamptz NULL,
attempts int NOT NULL DEFAULT 0,
created_at timestamptz NOT NULL DEFAULT now()
);
CREATE INDEX ON jobs (state, created_at);
Worker claim one job
BEGIN;
UPDATE jobs
SET state = 'processing', locked_at = now(), attempts = attempts + 1
WHERE id = (
SELECT id FROM jobs
WHERE state = 'queued'
ORDER BY created_at
FOR UPDATE SKIP LOCKED
LIMIT 1
)
RETURNING *;
COMMIT;
Explanation:
- The subquery selects one queued row and acquires a lock on it (
FOR UPDATE SKIP LOCKED). - The
UPDATE ... WHERE id = (subquery)then atomically marks that locked row asprocessingand returns it. - Other workers running the same statement will skip locked rows and claim different ones, avoiding contention.
Handling failures and retries
- If a worker crashes while processing, you need a re-queue mechanism. Options:
- Use a
locked_attimestamp and a background reaper that moves staleprocessingjobs back toqueuedafter a timeout. -
Use
attemptsto backoff or dead-letter after N retries. - Example pseudo-SQL:
UPDATE jobs
SET state = 'queued',
locked_at = NULL
WHERE state = 'processing'
AND locked_at < NOW() - INTERVAL '5 minutes';
When SKIP LOCKED is the right tool
- Best for work queues where ordering is not strictly required and high throughput with many workers is desired.
- Not suitable if strict ordering and fairness are required — SKIP LOCKED sacrifices strict ordering for throughput. If you need strict ordering, look at single-consumer partitions or other coordination mechanisms.
Summary
SKIP LOCKED gives you a simple, efficient, and robust primitive to implement worker queues in PostgreSQL. It avoids blocking contention and lets multiple workers claim tasks concurrently. Combined with a small reaper and careful retry logic, it becomes a production-ready foundation for many background processing needs.