Transactional Outbox

The transactional outbox is a reliability pattern in which a service writes state changes and outbound event records in the same local transaction, then a relay publishes those records to a message broker later. It addresses the dual-write problem: updating a database and publishing an event are two separate effects unless they are coordinated.

Transactional outbox designs appear in distributed services, telemetry platforms, industrial gateways, order systems and saga workflows. They do not make event delivery exactly once by themselves. They make the local state change and the intent to publish durable together.

Dual-Write Problem

Let the local state update be:

and the outbound event be:

A naive design performs:

If the service crashes between those effects, the database shows the state change but no event reaches downstream consumers. If the event publishes first and the commit fails, downstream systems may observe an event for state that does not exist.

Outbox Transaction

The outbox pattern writes both the business state and an outbox record:

inside one local transaction:

After commit, a relay reads O_i and publishes E_i. If the relay crashes, the outbox record remains available for retry.

Relay Lag

Let outbox arrival rate be:

and relay publish rate be:

Outbox backlog grows at:

when g_o is positive. Publication lag is part of the product behavior because consumers may act on stale state until the outbox drains.

Duplicate Delivery

A relay may publish an event and crash before marking the outbox row as sent. On restart, it may publish the same event again. Consumers therefore need idempotency, deduplication or a safe conflict rule.

If event identifier is:

then a consumer should apply side effects only if:

or otherwise prove that repeated processing is harmless.

Ordering

The design should state whether events are ordered globally, per aggregate, per partition or only best effort. Global order is expensive. Per-entity order is often enough, but it requires a stable key and relay behavior that does not reorder rows for that key.

An outbox can preserve the order in which records are committed locally. It cannot automatically impose a consistent order across unrelated services unless the architecture adds another ordering mechanism.

Relay Strategy

The relay can poll the outbox table, read a database log through change data capture or run inside a platform-specific streaming connector. Polling is simple and explicit, but it adds query load and polling delay. Log capture can reduce polling overhead, but it adds operational dependency on database log retention, connector offsets and replay tooling.

For relay polling interval:

the minimum publication lag includes:

before broker send time and consumer delay are even considered.

Retention and Cleanup

Outbox retention is a capacity issue. If retained row count is:

and average row size is:

then retained storage is:

Cleanup must not delete rows before they are safely published or before downstream replay requirements expire.

Failure Modes

Common failure modes include relay stopped but service still accepting writes, outbox table growth, duplicate event delivery, missing idempotency key, publish order mismatch, deleting rows too early, poison events that block the relay, unclear sent-state transitions and monitoring that tracks relay health but not oldest unsent event age.

The most common mistake is to treat the outbox as a queue without operational ownership. It is a reliability boundary and needs lag alerts, replay tools, retention rules and failure drills.

Worked Check

Suppose outbox records arrive at:

and the relay can publish:

The backlog growth rate is:

If current backlog is:

and the alert threshold is:

time to alert is:

The relay is already undersized. A larger table only delays visibility; it does not fix publication capacity.

Validation Evidence

Useful evidence includes crash tests between commit and relay publish, duplicate-publish tests, idempotency-key checks, relay restart tests, oldest-unsent event age, outbox backlog, relay throughput, poison-event handling, replay tooling, retention tests and downstream consistency checks.

A strong transactional-outbox review states exactly what is atomic, what is eventually published, what duplicates can happen and how operators know when publication lag has become a user-visible reliability problem.