Skip to content
sysgeeks

Transactional outbox

Recording outgoing messages in the same database transaction as the state change, then delivering them separately, to avoid the dual-write problem.

Reliability

Start the lessonExplain it in your own words

Learn it

0 of 1 checks done

  1. After a state change you often must tell another system: publish an event, enqueue a job, send an email. That's two writes to two systems, and no transaction spans both.

    Commit first and crash before publishing: the message is lost. Publish first and the commit fails: you announced something that never happened. This is the dual-write problem.

  2. The outbox makes the second write part of the first:

    1. In the same transaction as the state change, INSERT a row into an outbox table describing the message.
    2. Commit. The change and the intent to notify are durable together, or neither exists.
    3. A relay (a polling worker, or change-data-capture on the table) reads unsent rows, delivers them, and marks them sent.

  3. Check

    The relay delivers a message, then crashes before marking its row sent. What happens?

Quick reference

The same ideas, condensed for revision.

How it goes wrong

Relay without dedupe downstream
Redelivery after a relay crash duplicates effects.
Outbox growth
Sent rows are never deleted, and the relay's scan slows down.
Stuck relay
Messages accumulate unsent; alert on the age of the oldest unsent row.

Instead, consider

Publish after commit and reconcile
Occasional loss is detected and repaired by a sweep anyway.
Change data capture on business tables
Consumers can work from row changes directly, without explicit messages.
Jobs table as the queue
The consumer is your own worker; the 'outbox' row simply is the job.

In practice

Outbox table + polling relay
SELECT … FOR UPDATE SKIP LOCKED over unsent rows.
Debezium / logical replication
Stream outbox inserts from the database log.
Framework support
Many service frameworks ship outbox and inbox implementations.

It assumes

  • The state change lives in a database that can also hold the outbox table.
  • Consumers deduplicate by message ID or are idempotent.
  • Some delay between commit and delivery (typically sub-second to seconds) is acceptable.

Explain it in your own words

Write at least 60 characters (0 so far). Write it as you would say it in a design review. You will compare it against the points a strong answer makes.

Where you practise it

Further reading

Engineers describing it in systems they run.

  • Scaling Memcache at Facebook

    Meta (Facebook) · Rajesh Nishtala and others · Paper, Apr 2013

    The reference on running a look-aside cache hard: leases, invalidation from the commit log, failover without hammering the database, and consistency across regions.

Related concepts

  • Transactions

    Grouping several reads and writes so they take effect all together or not at all, isolated from concurrent work to a defined degree.

  • Message queues

    A durable buffer between producers and consumers that hands each message to one consumer at a time and redelivers it unless acknowledged.

  • Idempotency

    Designing an operation so that performing it twice has the same effect as performing it once, which is what makes retries safe.

  • Delivery guarantees

    At-most-once, at-least-once, and why 'exactly-once' is achieved by making duplicates harmless rather than by preventing them.