LLM Output Repair Pipelines: Fix Structure Without Hiding Failure

Sooner or later, a model returns JSON with a missing brace, a valid schema with an impossible value, or a beautifully written explanation where your application expected an enum.

The dangerous response is to keep patching strings until the parser stops complaining. That creates a system that converts visible failures into silent corruption.

An LLM output repair pipeline should be conservative. It can correct unambiguous formatting damage, but it must not invent business meaning, erase provenance, or turn repeated model failure into a successful status.

Start With the Strongest Contract

Repair begins before generation. Use native structured-output constraints when the model and provider support them. Define a strict schema with required fields, enums, numeric ranges, formats, and additionalProperties: false where appropriate.

Keep the schema smaller than your ambition. Deeply nested unions, dozens of optional fields, and overloaded string fields make both generation and validation harder. Split multi-stage work into separate typed outputs.

Validate more than syntax. A JSON parser can accept {"refund": -500}. Schema validation can reject the type or range, but business rules still need deterministic checks: the order exists, the currency matches, the requested operation is authorized, and totals reconcile.

Treat model output as untrusted input. Never pass generated strings directly into SQL, shell commands, HTML, or tool parameters without escaping, validation, and authorization.

Use a Repair Ladder

A repair ladder applies the least creative fix first:

1. parse the original response exactly;
2. apply deterministic normalization;
3. validate against the schema;
4. request one constrained model repair with error details;
5. revalidate the entire result;
6. quarantine or fail.

Deterministic normalization should be narrow: trim a code fence, remove a byte-order mark, or normalize a documented date representation. Avoid regex that guesses where objects begin and end or rewrites arbitrary quotes. Those tricks work on the sample that inspired them and fail on the next malformed payload.

If a model repair is necessary, provide the original output, schema, and validator errors. Instruct the repair step to return only the corrected object and preserve values unless a violation requires change. Use a separate trace span and model call so operators can see that the result was repaired.

One repair attempt is usually enough. Multiple retries increase latency and cost while making the final answer harder to attribute. If the same schema fails repeatedly, fix the prompt, schema, or model route instead of building a slot machine.

Distinguish Syntax From Semantics

Some errors are mechanically recoverable. A trailing comma has an obvious fix. A missing currency field does not. Choosing USD because most customers use it is not repair; it is an unauthorized business decision.

Classify failures:

• syntax errors prevent parsing;
• schema errors violate shape or type;
• semantic errors violate domain rules;
• authorization errors request forbidden actions;
• evidence errors lack support for a factual field.

Each class needs a different response. Syntax may be normalized. Schema errors may be regenerated. Semantic or authorization errors often require upstream context or human review. Evidence failures may require retrieval, not output editing.

Never let the repair model expand authority. If the original output proposed a read-only action, the repair step cannot “fix” it into a write. Re-run policy checks after repair because the corrected structure may expose a valid but forbidden operation.

Preserve the Audit Trail

Store the raw output, normalized output, validation errors, repair prompt version, repaired output, and final validation result. Redact sensitive data according to policy, but keep stable hashes so events can be correlated.

Mark repaired results in telemetry. Measure initial validity rate, repair success rate, failures by field, added latency, token cost, and downstream incident rate. A rising repair rate is a regression even if users still receive valid JSON.

Sample repaired results for review. The pipeline may achieve schema validity while subtly changing meaning. Compare fields before and after repair and alert on high-impact changes such as prices, permissions, account IDs, or external-action parameters.

Fail Safely

The final fallback depends on consequence. A formatting suggestion can return a partial result with a warning. A financial operation, access change, or external message should stop when required fields remain ambiguous.

Quarantine failed payloads with enough context for debugging, not indefinite raw-data retention. Give the queue an owner, age limit, and replay mechanism tied to the original idempotency key.

Most importantly, return an honest status. “Could not produce a validated operation” is better than sending a guessed object downstream and recording success.

Structured output reliability is not a parser problem. It is a contract-enforcement problem. Constrain generation, validate deterministically, repair once, check policy again, and preserve the evidence. Anything more magical is probably hiding the exact failure you need to see.