Skip to content
System Design

Idempotency

One-Liner

An operation is idempotent if it can be called multiple times with the same input and produce the same result, with no additional side effects after the first call.

What It Is

A property of an operation that ensures that repeating the operation does not change the outcome. For example, x = 5 is idempotent, while x = x + 1 is not. In the context of APIs, a DELETE request is idempotent, while a POST request to create a new resource is typically not.

Why It Exists

In distributed systems, failures and retries are inevitable. Idempotency is crucial for building reliable systems because it allows a client to safely retry a failed request without worrying about creating duplicate data or causing other unintended side effects.

How It Works

  • Client-side: The client generates a unique “idempotency key” for each request. The server stores a record of the keys it has already processed, and if it receives a request with a key it has seen before, it returns the original response without re-processing the request.
  • Server-side: The operation is designed to be naturally idempotent (e.g., using INSERT ON CONFLICT DO NOTHING in a database, or using a PUT request to update a resource to a specific state).

Tradeoffs

Pros

  • Greatly improves the reliability of a system by making retries safe.

Cons

  • Can add complexity to the application logic, as both the client and server need to be aware of idempotency.
  • Requires a mechanism to store and look up idempotency keys.

Failure Modes

  • Non-unique idempotency key: If the client generates a non-unique key, a new request could be incorrectly treated as a duplicate.
  • Purging idempotency keys too quickly: The server purges the record of a key, and then a late-arriving retry of that request is processed as a new request.

Interview Traps

  • Confusing idempotency with being stateless.
  • Not being able to explain how to make a non-idempotent operation (like creating a resource) idempotent (using an idempotency key).

Real-World Usage

  • Used by many payment APIs (like Stripe) to prevent duplicate charges.
  • Essential for reliable message queue consumers.

Anti-Patterns

  • Retrying non-idempotent operations without any mechanism to prevent duplicate processing.
  • Retries with Exponential Backoff
  • Message Queues
  • API Design