AI Agent Runbooks: Make Production Failures Boring

An AI agent can fail while every service dashboard is green. The model answered, the tool returned 200, and the queue drained—straight into the wrong customer account.

AI agent runbooks need to describe behavioral failure, not just infrastructure failure. The goal is to make a weird, high-context incident boring: stop the blast radius, preserve evidence, classify the fault, recover safely, and prevent recurrence.

What Should an Agent Runbook Cover?

A useful runbook starts with an observable symptom and ends with verified recovery. “Restart the worker” is not a runbook.

Failure class	Example	First safe action
model behavior	ignores constraint	disable affected workflow or route to review
tool contract	schema changed	pin or block the tool version
authorization	wrong scope used	revoke session and suspend writes
retrieval	stale policy ranked first	freeze index and use known-good snapshot
orchestration	task loops or duplicates	pause consumers and deduplicate by idempotency key
external dependency	partial write	reconcile remote state before retry

Every entry should name the owner, severity threshold, kill switch, evidence locations, rollback boundary, and verification query.

How Do You Design the Kill Switch?

The safest control is usually not “turn off AI.” It is a narrow switch for the risky capability: block email sends while leaving draft generation available, disable refunds while preserving read-only order lookup, or cap tool calls per run.

Build switches before launch and test them. An emergency flag buried in a deployment pipeline is useless when the incident commander lacks access.

capability: customer_email.send
mode: approval_required
scope: tenant:acme
reason: incident-2026-0814
expires_at: 2026-08-14T12:00:00Z

Time-bound changes reduce the chance that an emergency configuration becomes permanent folklore.

What Evidence Must You Preserve?

Capture the full execution chain: user intent, active policies, model request, model response, tool selection, validated arguments, tool result, retries, memory reads, retrieval chunks, approvals, and final user-visible state.

Do not dump secrets into the incident channel. Store sensitive payloads in the system designed for them and link stable trace IDs. Redaction should happen before telemetry leaves the trust boundary.

For every external mutation, record an idempotency key and remote identifier. Those two fields decide whether a retry repairs the run or doubles the damage.

How Do You Diagnose Without Guessing?

Walk the chain in order:

1. Did the system understand the correct user intent?
2. Did policy allow the action?
3. Did planning select an allowed tool?
4. Did validation reject unsafe arguments?
5. Did the remote system commit the change?
6. Did the agent interpret the result correctly?

This separates model error from application error. Teams love blaming “hallucination” because it sounds unavoidable. A missing authorization check or ambiguous tool schema is ordinary engineering debt.

Add a counterfactual replay. Run the same trace with the model response fixed, the retrieval snapshot fixed, or the tool mocked. The first layer that changes the outcome is where investigation should focus.

How Do You Recover Safely?

Never bulk-retry an unknown failure. First reconcile actual external state. A timed-out request may have succeeded remotely. Query by idempotency key, destination, timestamp, and amount before issuing a second mutation.

Define recovery states:

• contained: risky capability disabled;
• reconciled: intended and actual external state compared;
• repaired: incorrect state corrected where authorized;
• restored: capability re-enabled gradually;
• verified: success and error metrics remain normal through a defined window.

Use canaries when restoring. One tenant or one percent of runs is enough to prove the fix before opening the floodgates.

How Do You Test the Runbook?

Run game days. Inject a stale document, a malformed tool response, a duplicate queue message, an expired permission, and a provider timeout after remote commit. Ask an engineer who did not write the system to follow the runbook.

Measure time to containment, evidence completeness, unauthorized actions prevented, and whether recovery created duplicates. Rewrite any step that depends on tribal knowledge.

The best AI agent runbook turns “the agent did something weird” into a finite decision tree. Build capability-level controls, preserve replayable evidence, reconcile before retrying, and verify recovery against real external state.