Agent Tool Schema Evolution: Change APIs Without Breaking Autonomous Workflows

Humans complain when an API changes. Agents fail creatively.

Rename customer_id to account_id, change a default, or turn one string into an enum and a model may keep producing calls that look plausible but no longer mean the same thing. Tool schema evolution needs the discipline of database migrations, not the optimism of prompt editing.

Treat the Tool Contract as Production Code

Store each schema in source control with a stable tool identifier and explicit version. Review descriptions as behavior, because models use them to decide when and how to call the tool.

Validate more than syntax. A contract test should confirm required fields, bounds, enum values, formats, defaults, side effects, error shapes, idempotency behavior, and authorization requirements.

Descriptions should state what the tool does and what it does not do. “Updates customer” is dangerously vague. “Changes the shipping address for one verified order; does not change billing details or contact identity” gives the model and reviewer a real boundary.

Classify Compatible and Breaking Changes

Adding an optional field is often compatible. Making it required is breaking. Narrowing an enum, changing units, changing a default, altering side effects, or accepting a different identifier can break behavior even if validation still passes.

Be careful with optional fields. A new optional send_notification field with a default of true changes external behavior for every old caller. That is a breaking semantic change wearing a compatible-schema costume.

Version major behavior separately. Keep old and new tool definitions available during migration when risk justifies it: invoice.create.v1 and invoice.create.v2 are uglier than silent corruption and much cheaper than an incident.

Measure Calls Against Both Versions

Before switching execution, run shadow validation. Capture proposed calls from representative tasks, redact sensitive values, and validate them against the current and candidate schemas. Do not execute the shadow call.

Track invalid-call rate, field omissions, repair attempts, latency, token use, and task success. Include adversarial and ambiguous prompts, not just golden demos. A model may adapt to a renamed field while failing on a changed unit or default.

Replay real traces with fixed model versions where possible. Then test current models too, because schema wording interacts with model behavior.

Build an Adapter, Not a Hope-Based Migration

For simple changes, place a deterministic adapter between versions. It can rename fields, normalize formats, or reject ambiguous conversions. Every conversion should be observable and temporary.

function v1ToV2(input: V1): V2 {
  if (!/^cus_/.test(input.customer_id)) throw new Error("ambiguous customer id");
  return { account_id: input.customer_id, notify: false };
}

Never ask the model to infer a destructive conversion. If dollars became cents, the adapter should convert deterministically. If meaning changed, require a new plan or approval.

Roll Out Like an Infrastructure Change

Canary the new version by task type or tenant. Keep rollback fast. Dashboards should separate tool-validation failures from provider errors and business-rule rejections.

Long-running workflows need special handling. Persist the tool version with job state so a resumed job does not wake up into a new contract halfway through. Either finish on the old version or run an explicit state migration.

A tool schema is part API, part user interface for a probabilistic caller. Version it, test semantics, shadow real traffic, and preserve resumability. “The JSON still validates” is not a migration strategy.