Introduction to Structured Output Validation with LlamaIndex
Large language models (LLMs) are powerful text generators, but real-world applications rarely need free-form text. They need data — JSON objects, typed records, validated payloads that can flow into databases, APIs, and downstream business logic. Structured output validation is the practice of constraining an LLM's response to a predefined schema and verifying that the response conforms to that schema before it is used.
LlamaIndex provides first-class support for structured outputs through its StructuredLLMOutputParser, Pydantic-based output parsers, and the newer as_structured_llm API. This tutorial walks through everything you need to know to reliably extract structured data from LLMs using LlamaIndex.
What Is Structured Output Validation?
Structured output validation combines two concerns: constraining the model to produce output in a specific format, and validating that the output actually matches the expected schema. Without validation, an LLM might return a JSON object with a missing field, a string where an integer was expected, or an enum value that does not exist in your domain.
Validation typically relies on Pydantic, a Python data validation library. You define a Pydantic model that describes the shape of your data, and LlamaIndex uses that model both to prompt the LLM and to parse and validate its response.
Key Components
- Pydantic models — define the schema, field types, constraints, and validators.
- Output parsers — translate between the LLM's text output and Python objects.
- Structured LLM wrappers — bind a schema to an LLM so every call returns validated objects.
- Function calling / tool calling — many modern providers (OpenAI, Anthropic, Mistral) support native structured outputs via function-calling APIs.
Why Structured Output Validation Matters
Consider a customer support bot that extracts ticket information. If the LLM returns a free-form string, you have to write fragile regex parsers to pull out the customer name, priority, and issue category. If instead the LLM returns a validated SupportTicket object, your application code can treat the response as a typed Python object with guarantees about field presence and types.
The benefits compound in production:
- Type safety — downstream code receives objects with guaranteed field types.
- Reduced parsing errors — malformed JSON is caught at the boundary, not deep in business logic.
- Self-documenting prompts — the Pydantic schema itself communicates the expected shape to the model.
- Automatic retries — when validation fails, LlamaIndex can re-prompt the model with the error message.
- Composability — structured outputs plug cleanly into agents, query engines, and pipelines.
Setting Up Your Environment
Install LlamaIndex with the OpenAI integration and Pydantic v2:
pip install llama-index llama-index-llms-openai pydantic
Set your API key:
import os
os.environ["OPENAI_API_KEY"] = "sk-your-key-here"
For this tutorial we will use the OpenAI LLM, but the same patterns apply to Anthropic, Mistral, Ollama, and other providers that LlamaIndex supports.
Basic Example: Extracting a Person's Information
Let's start with a simple example. We want the LLM to read a short bio and return a structured Person object.
Step 1: Define the Pydantic Schema
from pydantic import BaseModel, Field
class Person(BaseModel):
"""Information about a person extracted from text."""
name: str = Field(..., description="The person's full name")
age: int = Field(..., description="The person's age in years", ge=0, le=150)
occupation: str = Field(..., description="The person's job title")
hobbies: list[str] = Field(
default_factory=list, description="List of the person's hobbies"
)
The description strings are not just documentation — LlamaIndex includes them in the prompt sent to the LLM, which dramatically improves extraction accuracy. The ge and le constraints enforce that age is between 0 and 150.
Step 2: Create a Structured LLM
from llama_index.llms.openai import OpenAI
llm = OpenAI(model="gpt-4o")
structured_llm = llm.as_structured_llm(Person)
The as_structured_llm method wraps the LLM so that every completion call is bound to the Person schema. Under the hood, this uses OpenAI's function-calling API to enforce the structure.
Step 3: Call the LLM and Get a Validated Object
from llama_index.core.llms import ChatMessage
bio = """
Sarah Chen is a 34-year-old software engineer at a fintech startup.
In her free time she enjoys rock climbing, playing the cello, and
baking sourdough bread.
"""
messages = [ChatMessage(role="user", content=f"Extract person info: {bio}")]
response = structured_llm.chat(messages)
person = response.raw
print(type(person)) # <class '__main__.Person'>
print(person.name) # Sarah Chen
print(person.age) # 34
print(person.occupation) # software engineer
print(person.hobbies) # ['rock climbing', 'playing the cello', 'baking sourdough bread']
The response.raw attribute contains the validated Pydantic object. If the LLM had returned invalid data — for example, a negative age — Pydantic would have raised a ValidationError before the object reached your code.
Using Output Parsers Directly
Sometimes you want more control than as_structured_llm provides. LlamaIndex exposes lower-level output parsers that you can use with any LLM call.
PydanticOutputParser
from llama_index.core.output_parsers import PydanticOutputParser
parser = PydanticOutputParser(output_cls=Person)
format_instructions = parser.get_format_instructions()
print(format_instructions)
The get_format_instructions() method returns a text block describing the expected JSON format. You append this to your prompt:
prompt = f"""
Extract the person's information from the following text.
{bio}
{format_instructions}
"""
response = llm.complete(prompt)
parsed = parser.parse(response.text)
print(parsed.name) # Sarah Chen
This approach works with any LLM, including local models that do not support function calling. The trade-off is that without native function-calling support, the model may occasionally produce malformed JSON, and you will need to handle parse errors.
Nested and Complex Schemas
Real-world data is rarely flat. LlamaIndex and Pydantic handle nested models, lists, optional fields, and enums with ease.
Defining a Nested Schema
from typing import Optional
from enum import Enum
class Priority(str, Enum):
LOW = "low"
MEDIUM = "medium"
HIGH = "high"
CRITICAL = "critical"
class ContactInfo(BaseModel):
email: Optional[str] = Field(None, description="Contact email address")
phone: Optional[str] = Field(None, description="Contact phone number")
class SupportTicket(BaseModel):
"""A customer support ticket extracted from a message."""
ticket_id: str = Field(..., description="Unique ticket identifier")
customer_name: str = Field(..., description="Name of the customer")
priority: Priority = Field(..., description="Urgency of the ticket")
summary: str = Field(..., description="One-sentence summary of the issue")
contact: ContactInfo = Field(..., description="Customer contact information")
tags: list[str] = Field(default_factory=list, description="Categorization tags")
Extracting a Nested Object
structured_llm = llm.as_structured_llm(SupportTicket)
message = """
Ticket #T-4821: John Martinez reports that the payment gateway is
returning 500 errors intermittently since this morning. This is
blocking customer checkouts. His email is john.m@example.com and
he can be reached at 555-0142. Please escalate — this seems critical.
"""
response = structured_llm.chat([
ChatMessage(role="user", content=f"Create a support ticket: {message}")
])
ticket = response.raw
print(ticket.ticket_id) # T-4821
print(ticket.priority) # Priority.CRITICAL
print(ticket.contact.email) # john.m@example.com
print(ticket.tags) # ['payment', 'gateway', 'checkout']
Notice how the enum Priority is automatically parsed into the correct member. Pydantic validates that the LLM's output matches one of the defined enum values; if the model returns "urgent" instead of "critical", validation fails.
Custom Validators
Pydantic v2 lets you define custom validation logic with the @field_validator decorator. This is invaluable for enforcing business rules that go beyond simple type checking.
from pydantic import field_validator
class ProductReview(BaseModel):
"""A product review extracted from user feedback."""
product_name: str = Field(..., description="Name of the product")
rating: int = Field(..., description="Rating from 1 to 5 stars")
review_text: str = Field(..., description="The review content")
would_recommend: bool = Field(..., description="Whether the user recommends it")
@field_validator("rating")
@classmethod
def validate_rating(cls, v: int) -> int:
if not 1 <= v <= 5:
raise ValueError(f"Rating must be between 1 and 5, got {v}")
return v
@field_validator("review_text")
@classmethod
def validate_review_length(cls, v: str) -> str:
if len(v.strip()) < 10:
raise ValueError("Review text must be at least 10 characters")
return v.strip()
If the LLM returns a rating of 7, the validator raises a ValidationError and LlamaIndex can be configured to retry the call with the error message appended to the prompt.
Handling Validation Failures with Retries
Even with function calling, models occasionally produce invalid output. LlamaIndex's OutputProgram and retry mechanisms help you recover gracefully.
Using the Structured Predict API
from llama_index.core.program import LLMTextCompletionProgram
program = LLMTextCompletionProgram.from_defaults(
output_parser=PydanticOutputParser(output_cls=ProductReview),
prompt_template_str=(
"Extract a product review from the following feedback:\n"
"{feedback}\n\n"
"{format_instructions}"
),
llm=llm,
verbose=True,
)
feedback = """
I bought the Acme Wireless Headphones last month. The sound quality
is amazing and the battery lasts forever. I'd give it 4 stars and
definitely recommend it to anyone looking for good value.
"""
review = program(feedback=feedback)
print(review.product_name) # Acme Wireless Headphones
print(review.rating) # 4
print(review.would_recommend) # True
With verbose=True, you can see the prompts and any retry attempts in the console. The program automatically re-prompts the LLM when validation fails, including the error message so the model can correct itself.
Structured Outputs in Query Pipelines
Structured outputs shine when integrated into LlamaIndex query pipelines. You can chain a retriever, a summarizer, and a structured extractor to build end-to-end data extraction workflows.
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
from llama_index.core.query_engine import CustomQueryEngine
# Load and index documents
documents = SimpleDirectoryReader("./data").load_data()
index = VectorStoreIndex.from_documents(documents)
# Create a structured extraction query engine
class FinancialMetric(BaseModel):
metric_name: str = Field(..., description="Name of the financial metric")
value: float = Field(..., description="Numeric value of the metric")
unit: Optional[str] = Field(None, description="Unit of measurement")
period: str = Field(..., description="Time period the metric applies to")
class FinancialReport(BaseModel):
company: str = Field(..., description="Company name")
metrics: list[FinancialMetric] = Field(
default_factory=list, description="Extracted financial metrics"
)
summary: str = Field(..., description="Brief summary of the report")
structured_llm = llm.as_structured_llm(FinancialReport)
retriever = index.as_retriever(similarity_top_k=5)
class FinancialExtractionEngine(CustomQueryEngine):
"""Extract structured financial data from indexed documents."""
def custom_query(self, query_str: str) -> FinancialReport:
nodes = retriever.retrieve(query_str)
context = "\n\n".join(n.get_content() for n in nodes)
prompt = f"Extract financial data from this context:\n\n{context}"
response = structured_llm.chat([
ChatMessage(role="user", content=prompt)
])
return response.raw
engine = FinancialExtractionEngine()
report = engine.query("What were Acme Corp's Q3 financial results?")
print(report.company)
for m in report.metrics:
print(f" {m.metric_name}: {m.value} {m.unit or ''} ({m.period})")
Structured Outputs with Function Calling Agents
When building agents, structured outputs let you define tools that return typed objects rather than raw strings. This makes agent reasoning more reliable.
from llama_index.core.agent import FunctionCallingAgent
from llama_index.core.tools import FunctionTool
class WeatherReport(BaseModel):
location: str = Field(..., description="City and country")
temperature_celsius: float = Field(..., description="Temperature in Celsius")
condition: str = Field(..., description="Weather condition description")
humidity_percent: int = Field(..., description="Humidity percentage", ge=0, le=100)
def get_weather(location: str) -> WeatherReport:
"""Get the current weather for a location."""
# In production, call a real weather API here
return WeatherReport(
location=location,
temperature_celsius=22.5,
condition="partly cloudy",
humidity_percent=65,
)
weather_tool = FunctionTool.from_defaults(fn=get_weather)
agent = FunctionCallingAgent.from_tools(
[weather_tool],
llm=llm,
verbose=True,
)
response = agent.chat("What's the weather like in Tokyo?")
print(response.response)
The agent calls get_weather, receives a validated WeatherReport object, and uses its fields to compose a natural-language answer. Because the tool returns a structured object, the agent never has to parse free-form text.
Best Practices
1. Write Descriptive Field Descriptions
Every Field should have a description. These descriptions are injected into the prompt and directly influence extraction quality. Be specific about format, allowed values, and edge cases.
# Bad
name: str = Field(...)
# Good
name: str = Field(
...,
description="The person's full legal name, including first and last name"
)
2. Use Enums for Constrained Values
When a field can only take a fixed set of values, use an enum. This prevents the model from inventing new categories.
class Sentiment(str, Enum):
POSITIVE = "positive"
NEUTRAL = "neutral"
NEGATIVE = "negative"
class SentimentAnalysis(BaseModel):
text: str = Field(..., description="The analyzed text")
sentiment: Sentiment = Field(..., description="Overall sentiment")
confidence: float = Field(..., description="Confidence score 0-1", ge=0, le=1)
3. Prefer Function Calling Over Prompt-Based Parsing
When your LLM provider supports function calling or structured output modes (OpenAI, Anthropic, Mistral), use as_structured_llm rather than PydanticOutputParser with manual format instructions. Function calling is significantly more reliable because the structure is enforced at the API level, not through prompt engineering.
4. Keep Schemas Focused
Avoid cramming dozens of fields into a single schema. If you need to extract a lot of information, break it into multiple smaller extraction calls. Smaller schemas produce more reliable results and are easier to debug.
5. Handle Errors Gracefully
Always wrap structured extraction in try/except blocks in production code. Even with function calling, network errors, rate limits, and occasional model quirks can cause failures.
from pydantic import ValidationError
try:
response = structured_llm.chat(messages)
person = response.raw
except ValidationError as e:
print(f"Validation failed: {e}")
# Fallback: retry, log, or return a default
except Exception as e:
print(f"LLM call failed: {e}")
6. Test with Edge Cases
Test your extraction pipeline with ambiguous inputs, missing information, and adversarial text. Pydantic validators will catch schema violations, but you should also verify that the model fills in fields correctly when the source text is unclear.
7. Use Optional Fields for Uncertain Data
If a field might not be present in the source text, make it Optional with a default of None. This prevents the model from hallucinating values just to satisfy a required field.
class MeetingNotes(BaseModel):
title: str = Field(..., description="Meeting title")
date: Optional[str] = Field(None, description="Meeting date in ISO format")
attendees: list[str] = Field(default_factory=list)
action_items: list[str] = Field(default_factory=list)
decisions: list[str] = Field(default_factory=list)
Conclusion
Structured output validation is one of the most important patterns for building reliable LLM applications. By combining Pydantic schemas with LlamaIndex's structured LLM wrappers, output parsers, and agent tools, you can transform unpredictable text generation into dependable, typed data extraction. Start with simple schemas and the as_structured_llm API, add custom validators as your business rules demand, and always handle validation errors gracefully. With these techniques, your LLM-powered features will be robust enough for production use, bridging the gap between the creative power of language models and the strict requirements of real software systems.