Getting Reliable Structured Outputs from LLMs: Validation, Retries and Constrained Decoding

The Problem of Semantic Integrity in LLM Pipelines

The current standard for structured output relies on enforcing a JSON schema or a Pydantic model at the generation layer. Whether utilizing OpenAI’s Response API or local inference engines like vLLM with xgrammar, the goal is to receive a deterministic object. However, a significant disconnect exists between syntactic validity and factual correctness.

Consider an invoice processing scenario. A model might correctly output a JSON object containing an invoice_id, a currency, and a total_cents value. If the model incorrectly calculates the total—interpreting "1,250.00" as 12,500 cents instead of 125,000 cents—the resulting JSON is syntactically perfect. It adheres to the schema, satisfies type constraints, and parses without error. Yet, the data is fundamentally wrong. Traditional JSON schemas cannot validate that a numeric value is derived accurately from the provided source text. This discrepancy represents a "semantic failure" that current generation-level constraints are ill-equipped to resolve.

A Layered Defense: Validation, Retries, and Decoding

To mitigate these risks, developers must implement a three-tier architectural approach: constrained decoding, contract validation, and source-text verification.

Constrained Decoding acts as the first line of defense. By masking tokens during the generation process, the model is physically prevented from emitting tokens that would violate the defined schema. This eliminates entire classes of errors, such as extraneous fields or invalid data types, before they ever reach the application logic.

Contract Validation follows, where the received object is compared against a hard-coded Pydantic contract. Using strict mode, developers can prevent "silent coercion"—where a model might provide a string where an integer is expected, and the system inadvertently converts it. By setting extra="forbid", the application ensures that the model does not inject hallucinated metadata that could downstream corrupt database entries.

Source-Text Verification serves as the final, and perhaps most important, gate. This involves a programmatic comparison between the model’s output and the original source document. For instance, if the model claims the total is 125,000 cents, the verification logic must cross-reference this against the verbatim text of the invoice. If the evidence does not support the claim, the result must be flagged as "rejected" rather than "accepted."

The Logic of Controlled Retries

Not every failure warrants a retry. A common pitfall in pipeline design is the "infinite retry loop," which consumes tokens, increases latency, and often fails to resolve the underlying issue. A professional-grade implementation must distinguish between transient errors and terminal failures.

A "shape error"—such as malformed JSON or a schema violation—is a candidate for a retry. In this instance, providing the model with specific, localized feedback regarding the failure can guide it toward a valid output. However, if the failure is semantic—for example, if the model misread a value that is clearly present in the source—a retry is unlikely to succeed. Repeating a request for a value the model has already hallucinated is a waste of resources. In such cases, the system should trigger a "needs_review" status, routing the document to a human operator for manual reconciliation.

The industry standard for retry budgets is generally capped at three attempts. This limit balances the need for recovery with the necessity of maintaining system throughput and cost control. Furthermore, developers must account for the SDK-level retries that often occur behind the scenes, such as exponential backoff for network timeouts or 429 rate-limiting errors. Failing to synchronize application-level retries with infrastructure-level retries can lead to an unpredictable explosion in request volume.

Strategic Implications for Data Extraction

The shift toward structured output is driving significant changes in how companies handle document intelligence. Historically, OCR (Optical Character Recognition) was the primary point of failure. Today, while OCR quality has improved, the "reasoning gap" provided by LLMs has introduced new risks.

Data from recent enterprise software deployments suggests that pipelines failing to implement source-verification layers experience an "accuracy drift" of 5% to 12% over time, as models tend to prioritize conversational fluency over mathematical precision. By enforcing strict contracts and implementing verification loops, organizations have reported reducing these error rates to under 0.5%.

Establishing a Robust Pipeline Workflow

To ensure production stability, the pipeline should follow a strict, deterministic sequence:

1. Preparation: Define a rigid Pydantic contract. Avoid optional fields where possible, as null values can hide missing data.
2. Generation: Utilize constrained decoding to ensure the output conforms to the JSON schema.
3. Completion Check: Before reading the object, verify the response status. If the model failed to complete or refused to generate, the pipeline must stop.
4. Verification: Execute cross-reference logic. Does the total match the source? Does the ID match the query?
5. Disposition:
   • Accepted: Data is committed to the downstream system.
   • Needs Review: Data is held for human intervention.
   • Rejected: System stops, logs the failure, and prevents further writes.

Conclusion

The reliability of LLM-based data extraction is not a feature provided by the models themselves, but an emergent property of the surrounding software architecture. By moving away from the assumption that a parsed JSON object is "correct," developers can build robust, audit-friendly systems. The future of AI integration lies in this shift: treating models as fallible, non-deterministic components that must be wrapped in rigorous, deterministic validation frameworks. Through the combined application of constrained decoding, contract-based validation, and evidence-based verification, the enterprise can harness the power of LLMs while maintaining the integrity of their data foundations.