Blog

Idempotency: the least conspicuous feature of a payments API

Networks are unreliable and clients retry. Why an idempotency key is not a detail but the foundation.

A request goes out and the response never comes back. Timeout. The client tries again. Has the payment now happened once, or twice? If an API cannot answer that question unambiguously, it is not fit for money. That holds for every payments API, whatever the provider and whatever the protocol.

The principle

An endpoint is idempotent when the same request, repeated any number of times, has the same effect as making it once. For GET that is a given. For POST /payments you have to build it.

The client sends a self-generated key with every write request:

POST /v1/payments
Idempotency-Key: 8f1c2a4e-3b7d-4c9e-a1f0-5d6e7f8a9b0c

The server remembers the result of the first processing under that key. Every repetition with the same key gets the same response, without executing the payment again.

What often gets missed

The key has to match the request. The same key with a different body is an error, not a retry. We check a hash of the body and answer with 422 when it differs.

Concurrent retries. Two identical requests arrive at the same moment. The key has to be reserved before processing — an INSERT ... ON CONFLICT DO NOTHING in the same transaction — otherwise both proceed.

Expiry. Keys do not live forever. Twenty-four hours is a good default; after that a repetition is a new request.

A fragment in Go

func (s *Service) CreatePayment(ctx context.Context, key string, req PaymentRequest) (Payment, error) {
    return s.idem.Run(ctx, key, req.Hash(), func(ctx context.Context) (Payment, error) {
        return s.payments.Create(ctx, req)
    })
}

The whole of the logic — reserve, execute, store the result, return it on a repeat — lives in one package that every write endpoint uses. No endpoint decides for itself whether it is idempotent.

Why this is the foundation

Retries are not an error case, they are the normal case: mobile networks, load balancers, timeouts, queues with at-least-once delivery. An API that does not recognise repetitions as repetitions moves the problem into the customer’s accounting department. It is more expensive there.

A conversation, not a newsletter

Let's talk about your system

If this article describes something you recognise, a conversation is the shortest route to an answer.

Let's talk