Bedrock Claude Can Stop Broken JSON—But It Cannot Stop Hallucinations

Amazon Bedrock can now force Anthropic Claude responses to match a JSON schema during generation. The feature eliminates a common class of production failures: malformed JSON, missing required fields, wrong field types and invalid tool parameters.

It does not guarantee that the values inside the valid structure are true. A model can return a perfectly formed customer record with the wrong account status, or call a tool with syntactically valid but operationally dangerous arguments. Structured outputs make the container reliable; they do not make the judgment reliable.

That boundary is the most important fact in AWS’s announcement. The feature is valuable precisely because it solves a narrow, expensive problem. Teams should adopt it without confusing schema compliance with factual accuracy.

What Bedrock now guarantees

AWS says structured outputs are available for all Anthropic Claude models on Amazon Bedrock where those models are offered. They work through the bedrock-runtime and bedrock-mantle endpoints.

Developers can use two mechanisms. JSON Schema output format constrains the response structure, while strict tool use constrains tool-call parameters to a declared schema. Required fields must appear, field types are enforced, enums remain within allowed values, and undeclared properties can be rejected.

The constraint happens during decoding. Bedrock compiles the schema into a grammar and prevents the model from producing tokens that would violate it. This differs from asking for JSON in a prompt and repairing the output afterward.

For applications, that means fewer parsing exceptions and fewer retries caused solely by invalid formatting. It can also reduce output tokens because the model is restricted to the requested fields instead of surrounding the result with explanations.

The mechanism deserves a closer look, because it explains both the feature’s strength and its limits. Constrained decoding works at the level of the token stream: at each step, the decoder masks out any candidate token that could not lead to a string conforming to the compiled grammar. The model’s probability distribution is effectively renormalized over the permitted continuations. This is why the guarantee is structural rather than persuasive. A prompt that says “respond only in JSON” is a suggestion the model may weigh against everything else it has learned; a grammar is a hard boundary the sampler cannot cross. The same property means the model never gets to “change its mind” mid-object and restart, which is what produces the truncated braces and stray prose that plague unconstrained generation.

That distinction matters for anyone who has maintained a repair loop in production. The loop is not merely inelegant; it multiplies cost and variance. Each retry is a fresh sample with a fresh chance of a different error, and the latency of a second or third generation often dominates the request. Removing the loop removes a source of nondeterminism that is hard to test against and harder to explain in an incident review.

How the compilation and cache work

When Bedrock receives a new schema, it validates the definition against its supported subset of JSON Schema Draft 2020-12. It then compiles a grammar artifact used during generation.

The first request may take longer because of that compilation step. AWS says compiled grammars are cached for 24 hours within the customer’s account, so later requests using the same schema should have minimal additional overhead.

Changing the schema’s structure invalidates the cache and triggers compilation again. Changing only names or descriptions does not. Teams with dynamically generated schemas should account for first-request latency and avoid unnecessary structural variation.

That cache behavior has an architectural implication that is easy to miss. Schemas should be treated as versioned artifacts, not as strings assembled on the fly from user input or request context. A team that generates a fresh schema per tenant, per locale or per feature flag will pay the compilation cost repeatedly and lose the benefit of warm caching. The practical pattern is a small, stable library of schemas checked into the codebase, reviewed like any other interface contract, and referenced by identifier at call time. If variation is genuinely needed, prefer varying values within a fixed structure over varying the structure itself.

The “supported subset” phrasing is also worth flagging. JSON Schema is a large specification, and constrained decoding engines typically implement a practical subset rather than every keyword and composition rule. Teams porting elaborate schemas from a post-hoc validator should test them against the service before assuming parity, and should keep the post-hoc validator in place as a second line of defense rather than removing it.

The failures it removes

Structured output is particularly useful for extraction, classification and tool orchestration.

An invoice pipeline can require vendor, amount, currency and due date fields. A support classifier can restrict priority to a fixed enum. An agent can call a weather or database tool only with the parameters that tool accepts.

Without constrained decoding, developers often build a repair loop: parse the text, detect an error, ask the model to fix it and retry. That loop adds latency, cost and another opportunity for inconsistent behavior. Bedrock can remove much of that plumbing when the failure is structural.

Strict tool schemas also reduce accidental interface errors in multi-step agents. If a tool expects a location string and a unit enum, Claude cannot invent an extra property or use an unsupported value when strict enforcement is enabled.

The tool-use case may be the most consequential. In a multi-step agent, a single malformed argument can cascade: a wrong type throws an exception that aborts the run, an invented property is silently ignored by a lenient handler, an out-of-enum value routes the request down a default branch nobody tested. Constraining the argument space at generation time turns these into impossible states rather than handled exceptions. It also makes the tool contract enforceable in a way that documentation never was, because the schema the model sees and the schema the runtime enforces are the same object.

The failures it does not remove

Schema validation cannot verify whether extracted facts match the source. A required date field can contain the wrong date. A sentiment enum can be valid but misclassified. A tool call can satisfy every type rule while targeting the wrong customer record.

Business rules often exceed JSON Schema. A refund amount may be numeric and positive yet exceed the user’s authorization. A transfer destination may be a valid string but belong to an untrusted account. A medical code may match the expected pattern while being unsupported by the clinical note.

The model can also omit nuance by placing a complex situation into an allowed category. Constrained decoding narrows the output space; it does not supply missing evidence or improve the model’s underlying knowledge.

Developers must therefore keep source-grounding checks, permission enforcement, business validation and human approval for consequential actions.

There is a subtler failure mode as well: the schema can create false confidence in reviewers. When every response parses and every field is populated, dashboards look clean, and the absence of exceptions is mistaken for correctness. A field that is required will always be filled, even when the source document never mentioned it, because the grammar leaves the model no option to return null unless null is explicitly permitted. Teams should decide deliberately which fields are nullable and treat an unexpectedly populated field as a signal to inspect, not a sign of health.

A production design that uses the feature correctly

Use structured outputs as one layer in a larger reliability system.

Define the smallest schema the downstream task needs. Reject additional properties. Use enums for closed sets and explicit required fields. Keep descriptions clear, but do not rely on descriptions as security policy.

After generation, validate business rules separately. Confirm identifiers against trusted systems, check numerical ranges, enforce user permissions and require human approval for irreversible actions. Log the source context, model version, schema version and tool result so failures can be audited.

For extraction, attach evidence spans or source references to important fields. For agents, separate read-only tools from state-changing tools and use narrower credentials for each action.

Finally, monitor semantic error rates even if JSON parsing errors fall to zero. A dashboard that celebrates perfect schema compliance while ignoring wrong values measures the wrong thing.

The logging point is not incidental. Because the schema version is now part of the contract, it belongs in the audit trail alongside the model version and the source context. When a downstream consumer reports a bad record, the first question will be which schema and which model produced it, and answering that from logs is far cheaper than reconstructing it from memory. Evidence spans serve a similar purpose on the semantic side: they give a reviewer something to check against, converting an unverifiable claim into a testable one.

Verdict

AWS has removed a frustrating source of production instability for Claude users on Bedrock. Generating schema-compliant JSON and strict tool inputs at decoding time is more reliable than prompt instructions and repair loops.

The feature should make extraction pipelines and agents simpler, faster and cheaper. It should not change the fundamental trust model. Valid structure is necessary for automation, but factual grounding, authorization and human accountability remain separate engineering problems.

Bedrock Claude can now stop broken JSON. It still cannot stop a well-formatted hallucination from reaching your application unless the rest of the system is designed to catch it.