Idempotency Key

An idempotency key is a stable request identifier used to recognize repeated attempts for the same logical operation. When the receiver sees the same key again, it returns the stored result or a controlled conflict instead of applying the side effect a second time.

The concept matters because retries are common exactly when systems are uncertain. A client may time out after the server has already created a reservation, written a command, accepted a payment, started a job or changed a configuration. Without a duplicate-control rule, the retry can convert an availability feature into a correctness failure.

Why It Exists

Idempotency is easy for pure reads and some replacement operations. It is harder for operations that create a new resource, decrement stock, reserve capacity, issue a command, send a message, actuate equipment or trigger a workflow. Those operations may be safe to retry only if the receiver can tell whether the retry is the same logical request.

An idempotency key is therefore part of the retry contract. The caller supplies a key that remains stable across attempts. The receiver stores the key, a payload fingerprint, the decision and enough response data to answer later retries consistently.

Key Scope

A key must be scoped to the operation boundary. A practical key can be modeled as:

The receiver should not treat the same raw string as globally meaningful unless that is an explicit design choice. Including client or tenant scope prevents one caller from blocking another caller with the same local identifier.

The key must also be bound to the intended payload. If the same key is reused with a different command body, the safe response is usually a conflict, not a second execution.

Duplicate Decision Rule

Let the stored payload fingerprint for key K be:

and the incoming payload fingerprint be:

If:

the request is a duplicate attempt for the same logical operation. The receiver should return the stored accepted, rejected or failed outcome according to the service contract.

If:

the request is a key reuse conflict. Executing it as a new command would hide an integration error and may corrupt state.

Replay Window

The receiver must retain key records long enough to cover late retries and network replays. A first retention screen is:

where T_ttl is the key time-to-live, T_caller is the caller-visible deadline, T_retry,max is the maximum retry horizon, T_network is delayed delivery allowance and T_clock covers timing uncertainty between systems.

Too short a window turns old retries into new operations. Too long a window increases storage cost and may make legitimate repeated operations harder to express. The design should define how clients create a new key when they truly intend a new command.

Storage Capacity

For accepted command rate:

and retention time:

the minimum stored-key population is:

Production designs add margin for bursts, delayed cleanup, retries, partition recovery and audit retention. If the key store saturates first, the idempotency mechanism can fail during the same incident it is meant to control.

Collision Risk

If keys are random with b bits of entropy and n active keys exist in the replay window, a birthday-bound screen for accidental collision is:

This does not cover malicious guessing, weak client generators or operational reuse of a fixed key. High-consequence systems should use authenticated clients, scoped keys, payload fingerprints and monitoring for abnormal key reuse.

Worked Example

A command API accepts:

The caller deadline is:

The maximum retry horizon is:

Network replay allowance is:

Clock and cleanup uncertainty is:

The retention window should be at least:

With a 20 percent storage margin:

During a degraded event, duplicate retry attempts occur for:

of commands. Duplicate attempt rate is:

Over ten minutes:

uncontrolled duplicate side-effect opportunities are:

The idempotency target for accepted duplicate side effects is not “small.” For the same key and same payload, it should be:

If active keys in the window are about:

and keys have 128 random bits, then:

The collision risk is negligible for accidental collisions, assuming the key generator is real and the service also checks payload fingerprints.

Relationship to Sequence Counters

A sequence counter proves order and continuity in a stream. An idempotency key proves that a retried command is the same logical operation. They can work together, but they are not substitutes.

A sequence counter can detect that message 105 arrived after message 104. It does not necessarily say whether command 105 is a duplicate of a command whose response was lost. An idempotency key can suppress duplicate side effects even when transport delivery, retry timing and response visibility are uncertain.

Relationship to Exactly-Once Claims

An idempotency key does not create magical exactly-once execution. It creates at-most-once side-effect acceptance for the key scope and retention window, provided the key record and state transition are committed consistently.

The dangerous gap is a partial commit. If the business state changes but the idempotency record is not stored, a retry may execute again. If the key record is stored but the state change fails, the receiver may report a success that never happened. The transaction boundary must cover both the side effect and the idempotency outcome, or the system needs a reconciled recovery workflow.

Validation Evidence

Useful evidence includes duplicate-request tests, payload-mismatch tests, timeout-after-commit tests, retry storm tests, key-store capacity checks, cleanup behavior, transaction-boundary proof, observability for key reuse and metrics that separate original commands from duplicate attempts.

Validation should include degraded service behavior. The worst duplicates often appear when a caller times out, a worker restarts, a queue redelivers a message or a client library retries after losing the response.

Common Mistakes

Do not generate a new key for every retry attempt. Do not store only the key without the payload fingerprint. Do not let key expiry be shorter than the retry horizon. Do not treat a duplicated success and a duplicated failure differently unless the contract says so. Do not rely on a sequence number alone when the user-visible operation can be retried after an ambiguous outcome.

A good idempotency-key design states the key scope, payload binding, replay window, retention capacity, transaction boundary, conflict response, observability and evidence needed before retries are enabled for state-changing operations.