Mehdi Akiki
Published on

What an Idempotency Key Is (and How to Build One)

Authors
  • Mehdi Akiki avatar
    Name
    Mehdi Akiki
    Twitter

Reference · Interrupted execution

You click "Pay". The request takes too long. The connection drops before the answer comes back.

Did the payment go through? You do not know. So you retry. And now you charged the customer two times.

This is the problem that an idempotency key solves.

The naive fix does not work

The first idea is "I will check before I write." Before creating the payment, ask the database if a similar one already exists.

But "similar" is hard to define, and there is a race: two retries can both check at the same time, both see nothing, and both create a payment. Checking is not enough. You need something that the system can recognize with certainty.

The idea

An idempotency key is a unique id that the client chooses and attaches to a write. It means: "this is the same operation, not a new one."

The client generates one key for one logical action (one payment), and reuses the same key on every retry of that action.

The server keeps a table of keys it has already processed. When a request arrives:

  • if the key is new, do the work, then save the key with the result.
  • if the key is already there, do not work again, just return the saved result.
// simplified. real code claims the key first with a unique constraint,
// so two retries cannot both pass this check.
async function pay(key: string, amount: number) {
  const seen = await db.idempotency.get(key);
  if (seen) return seen.result; // already done -> return the same answer

  const result = await charge(amount);
  await db.idempotency.put(key, { result }); // remember this key
  return result;
}

Now a retry with the same key is safe. The customer is charged one time, and every retry gets the same answer.

The details that matter

  • The client owns the key. It must be the same across retries, so generate it once, before the first try, not inside the retry loop.
  • Store the key and the result in the same transaction as the write. The clean trick is to insert the key first with a unique constraint, so two retries cannot both pass the check.
  • Give keys a scope and a lifetime. A key is usually unique per endpoint and per user, and you can expire old keys after a day or a week.

An idempotency key does not make your operation correct by magic. It makes it safe to repeat. That is exactly what you need in a world where networks time out and clients retry.

This is one of the building blocks behind keeping two systems in sync: reconciliation only works because the writes it repeats are idempotent.