Idempotency for Tool-Using AI Agents

An AI agent sends a refund request. The tool times out before returning a response. Did the refund fail, or did the payment provider complete it while the network dropped the acknowledgment?

If the agent simply retries, the customer may receive two refunds. The model did nothing irrational: it saw uncertainty and tried again. The system around the model failed to make repetition safe.

Idempotency is the engineering property that lets the same logical operation run more than once without producing additional side effects. It is essential whenever an agent can send email, create tickets, modify records, place orders, transfer money, or trigger deployments.

Separate Intent From Attempt

Treat the model’s request as an intent, not as a one-off HTTP call. Give each logical action a stable operation ID before executing it. Retries reuse that ID.

type AgentOperation = {
  operationId: string;
  conversationId: string;
  tool: string;
  normalizedArgs: unknown;
  status: "pending" | "succeeded" | "failed" | "unknown";
};

The operation ID should not be generated inside the retry loop. It can be derived from an approved plan step or created when the orchestrator first accepts the intent. A new user instruction gets a new ID; a transport retry does not.

Normalize arguments before comparing requests. Two payloads that differ only in key order or whitespace should not become separate financial actions. For sensitive tools, bind the ID to a hash of the normalized payload and reject reuse with different arguments.

Put a Ledger in Front of Side Effects

Persist operations before calling the external tool. The ledger becomes the source of truth for whether work is new, running, complete, or ambiguous.

async function executeOnce(op: AgentOperation) {
  const existing = await operations.get(op.operationId);

  if (existing?.status === "succeeded") return existing.result;
  if (existing?.status === "pending") throw new Error("operation in progress");

  await operations.insertPending(op);

  try {
    const result = await tools.call(op.tool, op.normalizedArgs, {
      idempotencyKey: op.operationId,
    });
    await operations.markSucceeded(op.operationId, result);
    return result;
  } catch (error) {
    await operations.markUnknown(op.operationId, serialize(error));
    throw error;
  }
}

Why mark a timeout as unknown instead of failed? Because a timeout describes what your client observed, not what the remote service did. The reconciliation path should query the provider using its own transaction reference before deciding whether to retry.

When a provider supports idempotency keys, pass your stable ID through. When it does not, create a uniqueness constraint in your own system or use a read-before-write strategy with a business identifier. Neither is perfect under every race condition, so document the guarantee each adapter can actually provide.

Design Tool Contracts for Safe Retries

A strong tool schema exposes the fields needed for deduplication and recovery. A send_invoice tool should accept a stable invoice ID, not only an amount and email address. A create_issue tool should return the canonical issue ID. A deployment tool should expose status lookup by release ID.

Classify tools by effect:

• Read-only calls can usually retry automatically.
• Idempotent writes can retry with the same operation key.
• Non-idempotent writes require reconciliation or human approval.
• Irreversible operations deserve a confirmation boundary and a durable receipt.

Do not let the model invent retry policy in prose. The orchestrator should enforce limits, backoff, timeout handling, and escalation. The model can decide what the user wants; deterministic code should decide whether an uncertain side effect is safe to repeat.

Test the Ambiguous Moments

Happy-path tests miss the failures that matter. Inject a timeout after the provider commits but before your client receives the response. Kill the worker after the external call and before the ledger update. Deliver the same queue message twice. Run two workers against the same operation ID.

Your assertions should verify that the external effect occurs once, the ledger converges to a final state, and operators can reconstruct what happened. Keep raw provider IDs and timestamps in the audit record, while redacting secrets and unnecessary personal data.

The key metric is not merely tool-call success rate. Track duplicate attempts prevented, operations stuck in unknown, reconciliation latency, and manual interventions. Those numbers reveal whether the safety system works under real uncertainty.

An agent becomes operationally trustworthy when repetition is boring. Stable intent IDs, a durable ledger, provider-level idempotency, and explicit reconciliation turn retries from a gamble into a controlled mechanism.