You submit a payment. The page waits, then reports a connection error. Did the payment fail, or did it succeed while the confirmation failed to reach you?
The same uncertainty appears when a workflow publishes an article, sends a message, creates an order, or updates a record. An action can succeed even when the person or process that requested it never receives a clear answer.Idempotency means that performing the same operation more than once has the same intended effect as performing it once.
In workflows, it helps repeated attempts resolve to one intended outcome rather than duplicate consequences.
The reason it matters is straightforward: uncertainty about an outcome is not permission to create that outcome again.
Repeated Requests Are Not Always Repeated Intentions
A retry usually means, “Please finish or confirm what I already asked you to do.” It does not necessarily mean, “Do it a second time.”
Systems encounter repeated attempts for ordinary reasons:
- A person clicks again after a slow response.
- A client automatically retries after losing its connection.
- A background job restarts after an interruption.
- A message is delivered again because its acknowledgment was lost.
- An operator resumes a workflow whose last outcome is unclear.
Whether another attempt should produce another consequence depends on the intention behind it. Two identical purchases may be intentional. Two requests for the same purchase may not be.
A reliable workflow therefore needs more than a comparison of request contents. It needs a way to identify the logical operation those requests belong to. This distinction is central to retry safety: recovering from an interruption without turning recovery into duplication.(coming soon)
What Idempotency Protects
Some operations are naturally idempotent with respect to the state they change. Others need an explicit mechanism to prevent repeated effects.
| Instruction | Effect of repeating it |
|---|---|
| Set the subscription status to “paused.” | The subscription remains paused. |
| Extend the subscription by one month. | Each execution adds another month unless duplicate protection intervenes. |
The surrounding behavior matters too. If every “set to paused” request sends another email, the status update may be idempotent while the notification behavior is not.
Idempotency must be evaluated at the boundary where the relevant consequence occurs—not just where a value is stored. Updating a database row once does not establish that a related payment, publication, or message was also handled once.
Idempotency does not require every response to be identical. A first deletion might report success, while a later attempt reports that the record is already absent. The intended effect remains the same: the record is absent.
Nor does idempotency prohibit incidental activity such as recording each request in an access log. The important question is which effects the operation promises not to repeat.
How Idempotency Keys Give Retries a Shared Identity
An idempotency key identifies one logical operation across multiple attempts. It is commonly used for actions that create something new, where repeating the underlying instruction would otherwise create another result.
For example, a payment workflow might work as follows:
- The checkout system creates a key for one intended payment and preserves it for recovery.
- It sends the payment request with that key.
- The payment service associates the key with the operation and its outcome.
- If the response is lost, checkout retries with the same key.
- The service recognizes the existing operation instead of treating the retry as a new payment.
The key means, in effect, “This request belongs to the operation we already started.”
A retry keeps the operation’s key. A genuinely new operation gets a new one. Generating a fresh key for every attempt defeats the protection. Reusing one key for unrelated work can cause legitimate new work to be mistaken for a duplicate.
A key also needs supporting behavior on the receiving side. Adding a key to a request does nothing if the recipient does not recognize and enforce it.
The recipient should validate that a reused key still refers to the same relevant request details. A changed amount, destination, or payload should produce a clear conflict or another documented response—not silently alter the earlier operation.
Failure handling also needs a defined contract. Some services retain particular error results under a key; others permit another execution after certain failures. Clients need to understand those rules rather than assume that every repeated request behaves alike.
Duplicate Protection Must Handle Concurrent Requests
A simple check can look sufficient: if this operation has not been processed, process it.
But two requests can arrive almost simultaneously. Both may check before either records completion. Without coordination, both can proceed.
The system needs an atomic way to claim an operation’s identity, often through a database uniqueness constraint or another coordination mechanism. Competing requests must not independently conclude that each is the first.
Claiming an identity is only part of the design. The workflow also needs to distinguish meaningful execution states:
- In progress: the operation has been accepted and may still be executing.
- Completed: the intended effect and its result are established.
- Known failure without effect: the system has evidence that the intended consequence did not occur.
- Outcome unknown: the available evidence cannot establish whether the consequence occurred.
A duplicate arriving during execution might wait, receive an in-progress response, or be directed to a status check. It should not automatically start a second execution.
Similarly, an expired worker lease or an old “in progress” record does not, by itself, prove that an external action never happened. Recovery must account for what the earlier worker may already have done.
Partial Completion Is Where Recovery Gets Difficult
Consider a publishing workflow that:
- Creates an article on an external platform.
- Records the platform’s article identifier locally.
- Reports success to the operator.
The platform creates the article, but the workflow stops before saving its identifier. On restart, the local system sees no completion record. Publishing again could create a duplicate.
This is partial completion: a consequence exists, but the workflow’s evidence is incomplete. A local idempotency record alone cannot eliminate the gap between an external effect and the local record of that effect.
Depending on the destination’s capabilities, recovery may require:
- A destination-supported idempotency key that can be reused safely.
- A stable operation reference accepted by the destination and available for later lookup.
- A reconciliation process that checks for an existing result.
- A deliberate pause for human review when the outcome cannot be established safely.
A lookup can help, but it is not always conclusive. A recently created result may not yet appear in a search or listing. Its absence from that view does not necessarily establish that creation failed.
A transactional outbox can address a related local gap by recording a state change and pending delivery work in one database transaction. It does not, by itself, guarantee that an external recipient applies the effect only once. Delivery and recipient-side effect handling still need to work together.
This is why broad claims of “exactly once” need a stated boundary. A guarantee inside one system does not automatically extend to every external consequence.
A Timeout Does Not Establish Failure
A workflow timeout establishes that a particular component stopped waiting within a particular period. It does not necessarily establish that the work stopped.
The original request may still be running. It may already have succeeded. Its response may arrive later, or never arrive at all.
Before retrying a consequential action, a workflow should determine:
- Whether the original operation’s status can be checked.
- Whether the retry can use the same operation identity.
- What the destination does if both attempts arrive.
- How a late result will be associated with the original operation.
- Whether cancellation was merely requested or actually confirmed.
When the outcome is unknown, preserve that uncertainty. Labeling it “failed” may encourage an unsafe retry.
The interface should reflect the distinction too. “Confirmation unavailable; checking the original request” communicates a different situation from “Nothing happened; try again.” Clear status information is part of inspectability and recoverability, not merely a technical detail hidden behind the interface.
Idempotency Has Scope, Expiration, and Limits
Duplicate protection needs a defined scope. A service may associate keys with an account, an endpoint, or a particular operation type. It may retain them only for a documented period.
Those boundaries matter. If a retry arrives after the deduplication record has expired, the service may treat it as new work. The recovery period a workflow relies on should fit the protection its destination actually provides.
Changed intentions need separate treatment as well. Revising a payment amount or changing an article’s destination is not simply another attempt at an unchanged request. The workflow must determine whether it is handling a correction, a replacement, or a new operation—and what to do about any earlier effect.
Where human approval is involved, this also raises the issue of stale decisions. An earlier approval may cover one artifact, destination, or action, but not a later revision.
Operation identity is not unrestricted authority. Idempotency helps prevent duplicate effects; it does not establish permission, validate changed instructions, or undo an action that has already happened.
A Practical Review for Safe Continuation
When reviewing a workflow, start with its consequences rather than its retry settings:
- Identify the effect. What can this action create, send, publish, charge, delete, or change?
- Define the operation. How does the system distinguish another attempt from genuinely new work?
- Check concurrency. What happens when duplicate attempts arrive at the same time?
- Locate the evidence. Where are operation identity, status, and results durably recorded?
- Examine interruption points. What happens if execution stops after an external effect but before local confirmation?
- Verify the limits. Which effects are covered, and how long does protection remain available?
- Provide a recovery path. Can a person investigate an uncertain outcome without repeating the action?
Test these conditions deliberately: simultaneous requests, lost responses, worker restarts, changed payloads, and retries after records expire. A successful first attempt tells little about how a workflow behaves during recovery.
Not every operation needs elaborate infrastructure. Setting a local preference is different from charging a customer or publishing publicly. The safeguards should match the consequence, with clear workflow handoffs when automated recovery cannot establish a safe next step.
Repetition is a normal part of distributed systems and ordinary human recovery. The aim is not to prevent every repeated request. It is to let someone say, “Try that again,” without accidentally saying, “Make it happen twice.”