Mehdi Akiki
Published on

The Outbox Pattern: Send Events Without Losing Them

Authors
  • Mehdi Akiki avatar
    Name
    Mehdi Akiki
    Twitter

Reference · Interrupted execution

Your service does two things when a customer is created: it saves a row in the database, and it publishes an event to a queue so other systems can react.

await db.customers.insert(customer);
await queue.publish("customer.created", customer); // two separate writes

This looks normal. It is also broken, and the bug is hard to see.

The dual-write problem

These are two different systems, with no shared transaction. So one can succeed and the other can fail.

  • If the insert works but the publish fails, the customer exists but no event is sent. Other systems never learn about it.
  • If you publish first and then the insert fails, you sent an event for a customer that does not exist.

You cannot fix this by changing the order. Whatever you do first can succeed while the second one fails. No order is safe, because there is no transaction across the two.

The fix: write the event into the database

The outbox pattern turns two writes into one. Instead of publishing to the queue directly, you write the event into an outbox table, in the same database transaction as the data:

-- one transaction, one database
BEGIN;
  INSERT INTO customers (id, name) VALUES (42, 'Sam');
  INSERT INTO outbox (type, payload)
  VALUES ('customer.created', '{"id":42,"name":"Sam"}');
COMMIT;

Now both rows are saved together, or neither is. The database transaction gives you the guarantee that the queue could not.

Then a separate, simple process does the publishing. It reads unsent rows from the outbox, sends them to the queue, and marks them as sent:

loop:
  rows = SELECT * FROM outbox WHERE sent = false ORDER BY id
  for each row: publish(row); mark it sent

If this process crashes after publishing but before marking a row as sent, it will publish that row again on the next run. So the outbox gives you at-least-once delivery, not exactly-once. The consumer must be ready for duplicates, which is why it should use an idempotency key.

Why it matters

The outbox pattern is how you make sure that "the data changed" and "an event was sent" are always true together. No lost events, and no events for data that does not exist.

It is one half of keeping two systems in sync: the outbox makes your event stream reliable, and reconciliation catches the rare cases that still slip through.