The transactional outbox: publishing events without dual writes

Every event-driven system hits the same problem sooner or later. When an order is created, a service has to do two things: write the order to the database and publish an event so billing, inventory and notifications can react. Those are two operations on two different systems, and no transaction covers both.

The dual-write problem

Consider the most common sequence:

await db.orders.insert(order);
await broker.publish("order.created", order); // what if this fails?

If publishing fails after the insert, the order exists but no other service knows about it. Reversing the order doesn't help: if publishing succeeds and the insert fails, consumers react to an order that doesn't exist. Retries reduce the probability, but a process can die exactly between the two lines.

This problem doesn't show up in local tests. It shows up in production, rarely, and produces inconsistencies that are hard to investigate.

The outbox idea

The outbox pattern replaces two writes on different systems with two writes on the same database, in the same transaction. Instead of publishing directly, the service writes the event to an outbox table alongside the state change. A separate process, the relay, reads that table and publishes to the broker.

CREATE TABLE outbox (
  id            bigserial PRIMARY KEY,
  aggregate_id  text        NOT NULL,
  topic         text        NOT NULL,
  payload       jsonb       NOT NULL,
  created_at    timestamptz NOT NULL DEFAULT now(),
  published_at  timestamptz
);
CREATE INDEX outbox_unpublished ON outbox (id) WHERE published_at IS NULL;

await db.transaction(async (tx) => {
  await tx.orders.insert(order);
  await tx.outbox.insert({ aggregateId: order.id, topic: "order.created", payload: order });
});

Now either both rows are written or neither is. The event can't be lost, and there's no event without an order.

The relay

The relay reads unpublished events in order, publishes them and marks them as published. With PostgreSQL, FOR UPDATE SKIP LOCKED lets you run more than one instance without two of them taking the same batch.

async function relayBatch() {
  await db.transaction(async (tx) => {
    const batch = await tx.query(
      `SELECT id, topic, payload FROM outbox
        WHERE published_at IS NULL
        ORDER BY id
        LIMIT 100
        FOR UPDATE SKIP LOCKED`,
    );
    for (const event of batch.rows) {
      await broker.publish(event.topic, event.payload, { messageId: String(event.id) });
    }
    await tx.query(`UPDATE outbox SET published_at = now() WHERE id = ANY($1)`, [batch.rows.map((e) => e.id)]);
  });
}

There are two common ways to feed the relay:

• Polling, as in the example: simple, no extra dependencies, latency equal to the polling interval.
• Change data capture (CDC), reading the database's replication log, for example with PostgreSQL logical replication. Lower latency and less query load, at the cost of more infrastructure.

For most systems, starting with polling is the right call.

At least once, not exactly once

The relay can publish an event and crash before marking it as published. On the next run it publishes again. The outbox guarantees at-least-once delivery, and that's deliberate: losing an event is far worse than delivering it twice.

The consequence is that consumers must be idempotent. The most direct approach is to record processed message IDs in the same transaction as the effect:

CREATE TABLE processed_messages (
  consumer    text   NOT NULL,
  message_id  text   NOT NULL,
  PRIMARY KEY (consumer, message_id)
);
-- In the consumer's transaction: INSERT … ON CONFLICT DO NOTHING.
-- If no row was inserted, the message was already processed and the effect is skipped.

Ordering

When event order matters, it usually matters per aggregate: events for the same order must arrive in sequence, but different orders can be processed in parallel. Brokers that partition by key preserve that order when the partition key is the aggregate_id. Avoid requiring global ordering: it limits parallelism for no good reason.

Table maintenance

The outbox table grows continuously. Delete, or move to cold storage, events published longer ago than the period in which you might need to replay them. Also monitor the age of the oldest unpublished event: if it grows, the relay has stopped or can't keep up, and that's one of the most useful alerts in the system.

Summary

The transactional outbox solves a problem that retries and care can't: the impossibility of writing atomically to two systems. With an outbox table, a simple relay and idempotent consumers, events stop getting lost and systems stop drifting apart silently.