Introduction to Structured Output Validation with Pydantic AI
Large language models (LLMs) are powerful, but their outputs are inherently unstructured text. When you build production applications, you need reliable, predictable, and validated data that your downstream code can safely consume. This is where Pydantic AI shines — it combines the strengths of Pydantic's battle-tested validation framework with a modern agent framework designed specifically for LLM applications.
In this guide, you'll learn what structured output validation is, why it matters in the age of LLMs, how to implement it with Pydantic AI, and the best practices that will keep your applications robust in production.
What Is Structured Output Validation?
Structured output validation is the process of constraining an LLM's response to a specific schema and then validating that the response conforms to that schema before it is used. Instead of receiving free-form text and parsing it with fragile regular expressions, you define a typed model upfront, and the framework guarantees that the output matches that model.
Pydantic AI leverages Pydantic, the most widely used data validation library in the Python ecosystem. By defining Pydantic models as the expected output type of an agent, you get:
- Type safety: Fields are validated against their declared types.
- Automatic retries: If the LLM produces invalid output, Pydantic AI can retry with validation feedback.
- Tool-friendly design: The same validation logic applies to tool calls and agent responses.
- IDE support: Autocompletion and static analysis work because outputs are typed objects, not dictionaries.
Why Structured Output Validation Matters
Consider a scenario where you ask an LLM to extract customer information from an email. Without structured validation, you might receive something like:
"The customer's name is Jane Doe, she's 34 years old, and her email is jane@example.com."Parsing this with string manipulation is error-prone. What if the model forgets the email? What if the age is written as "thirty-four"? What if the response includes extra commentary that breaks your parser? Structured output validation solves all of these problems by:
- Forcing the model to produce JSON conforming to your schema.
- Validating each field against rules you define (type, format, constraints).
- Providing clear error messages when validation fails, enabling automatic correction.
- Ensuring your application code receives a trustworthy Python object, not a string you have to guess about.
In production systems, this translates directly to fewer bugs, less defensive coding, and higher reliability when integrating LLM outputs into databases, APIs, or business logic.
Getting Started with Pydantic AI
First, install Pydantic AI and your preferred LLM provider's integration. Pydantic AI supports multiple providers including OpenAI, Anthropic, Gemini, Groq, and others.
pip install pydantic-ai
pip install openai # or anthropic, etc.Set your API key as an environment variable:
export OPENAI_API_KEY="your-api-key-here"Now let's build a simple agent that returns structured output.
Defining Your First Structured Output Model
The core idea is to define a Pydantic model that represents the structure you want the LLM to produce. Pydantic AI uses this model both to instruct the model and to validate the response.
from pydantic import BaseModel, Field, EmailStr
from pydantic_ai import Agent
class CustomerInfo(BaseModel):
"""Structured representation of customer information extracted from text."""
name: str = Field(description="The customer's full name")
age: int = Field(description="The customer's age in years", ge=0, le=150)
email: EmailStr = Field(description="The customer's email address")
is_vip: bool = Field(default=False, description="Whether the customer is a VIP")
# Create an agent that returns CustomerInfo
agent = Agent(
"openai:gpt-4o",
output_type=CustomerInfo,
system_prompt=(
"You are a customer information extraction assistant. "
"Extract structured customer data from the provided text. "
"If a field is not present in the text, make a reasonable inference "
"or use the default value."
),
)
result = agent.run_sync(
"Hi, I'm Jane Doe. I just turned 34. You can reach me at jane@example.com. "
"I've been a premium member for 5 years!"
)
customer = result.output
print(f"Name: {customer.name}")
print(f"Age: {customer.age}")
print(f"Email: {customer.email}")
print(f"VIP: {customer.is_vip}")
print(f"Type: {type(customer)}")When you run this, result.output is not a string or a dictionary — it is a fully validated CustomerInfo instance. The Field descriptions serve double duty: they document your model for developers and guide the LLM toward producing the correct values.
How Validation and Retries Work
One of the most powerful features of Pydantic AI is its automatic retry mechanism. When the LLM produces output that fails validation, Pydantic AI does not simply raise an error. Instead, it feeds the validation error back to the model and asks it to correct the output.
Consider a model with strict constraints:
from pydantic import BaseModel, Field, conint
from pydantic_ai import Agent
class ProductReview(BaseModel):
product_name: str = Field(min_length=1, description="Name of the product")
rating: conint(ge=1, le=5) = Field(description="Rating from 1 to 5")
summary: str = Field(max_length=200, description="Brief review summary")
would_recommend: bool = Field(description="Whether the reviewer recommends it")
agent = Agent(
"openai:gpt-4o",
output_type=ProductReview,
system_prompt="Extract a structured product review from the user's text.",
retries=3, # Allow up to 3 retry attempts on validation failure
)
result = agent.run_sync(
"The Acme Wireless Headphones are fantastic! I'd give them 4 stars. "
"Great sound quality and comfortable. Definitely recommend."
)
review = result.output
print(review.model_dump_json(indent=2))If the model initially returns a rating of 7 (outside the 1-5 range), Pydantic AI intercepts the validation error, sends it back to the model with the error message, and the model gets another chance to produce valid output. This dramatically improves reliability compared to naive JSON parsing approaches.
Custom Validators
Pydantic supports custom validators using the @field_validator decorator. These run as part of the validation pipeline, and if they fail, the error is also fed back to the LLM for correction.
from pydantic import BaseModel, Field, field_validator
from pydantic_ai import Agent
class MeetingSchedule(BaseModel):
title: str = Field(description="Meeting title")
day: str = Field(description="Day of the week")
start_time: str = Field(description="Start time in 24-hour HH:MM format")
duration_minutes: int = Field(description="Duration in minutes", gt=0)
@field_validator("day")
@classmethod
def validate_day(cls, v: str) -> str:
valid_days = {"monday", "tuesday", "wednesday", "thursday",
"friday", "saturday", "sunday"}
normalized = v.strip().lower()
if normalized not in valid_days:
raise ValueError(
f"'{v}' is not a valid day. Must be one of: {sorted(valid_days)}"
)
return normalized.capitalize()
@field_validator("start_time")
@classmethod
def validate_time_format(cls, v: str) -> str:
parts = v.split(":")
if len(parts) != 2:
raise ValueError("Time must be in HH:MM format")
hours, minutes = parts
if not (0 <= int(hours) <= 23 and 0 <= int(minutes) <= 59):
raise ValueError("Invalid time values")
return f"{int(hours):02d}:{int(minutes):02d}"
agent = Agent(
"openai:gpt-4o",
output_type=MeetingSchedule,
system_prompt="Extract meeting schedule information from the user's text.",
)
result = agent.run_sync(
"Let's schedule the project kickoff for next Tuesday at 2:30 PM, "
"and we'll need about 90 minutes."
)
print(result.output.model_dump_json(indent=2))Notice how the custom validators normalize the data (capitalizing the day, zero-padding the time) in addition to validating it. This means your downstream code always receives clean, consistent data.
Nested Models and Complex Structures
Real-world applications rarely need just a flat object. Pydantic AI fully supports nested models, lists, and complex types. The LLM is guided by the full schema, including nested structures.
from typing import List
from pydantic import BaseModel, Field
from pydantic_ai import Agent
class Ingredient(BaseModel):
name: str = Field(description="Name of the ingredient")
amount: str = Field(description="Amount with unit, e.g., '2 cups'")
optional: bool = Field(default=False, description="Whether it's optional")
class RecipeStep(BaseModel):
step_number: int = Field(description="Sequential step number", ge=1)
instruction: str = Field(description="What to do in this step")
duration_minutes: int = Field(default=0, description="Estimated time", ge=0)
class Recipe(BaseModel):
title: str = Field(description="Recipe name")
servings: int = Field(description="Number of servings", ge=1, le=100)
ingredients: List[Ingredient] = Field(description="List of ingredients")
steps: List[RecipeStep] = Field(description="Ordered preparation steps")
total_time_minutes: int = Field(description="Total estimated time", ge=0)
agent = Agent(
"openai:gpt-4o",
output_type=Recipe,
system_prompt="You are a helpful cooking assistant. Extract structured recipes.",
)
result = agent.run_sync(
"Chocolate Chip Cookies: Makes 24 cookies. You need 2 cups flour, "
"1 cup butter (optional: use margarine instead), 3/4 cup sugar, "
"2 eggs, and 2 cups chocolate chips. "
"Step 1: Cream butter and sugar for 5 minutes. "
"Step 2: Mix in eggs, then dry ingredients, 3 minutes. "
"Step 3: Fold in chocolate chips, 2 minutes. "
"Step 4: Bake at 375F for 12 minutes. Total time about 25 minutes."
)
recipe = result.output
print(f"Recipe: {recipe.title} ({recipe.servings} servings)")
print(f"Total time: {recipe.total_time_minutes} minutes")
print(f"Ingredients: {len(recipe.ingredients)} items")
for ing in recipe.ingredients:
status = "(optional)" if ing.optional else ""
print(f" - {ing.amount} {ing.name} {status}")
print(f"Steps: {len(recipe.steps)}")
for step in recipe.steps:
print(f" {step.step_number}. {step.instruction} ({step.duration_minutes} min)")The nested structure is fully validated. If the LLM forgets to include the step_number field or provides a non-integer for servings, validation fails and the retry mechanism kicks in.
Using Enums for Constrained Values
When you want the LLM to choose from a fixed set of values, Python's Enum combined with Pydantic is the ideal tool. This prevents the model from inventing arbitrary categories.
from enum import Enum
from typing import List, Optional
from pydantic import BaseModel, Field
from pydantic_ai import Agent
class Priority(str, Enum):
LOW = "low"
MEDIUM = "medium"
HIGH = "high"
CRITICAL = "critical"
class TicketCategory(str, Enum):
BUG = "bug"
FEATURE = "feature"
SUPPORT = "support"
BILLING = "billing"
class SupportTicket(BaseModel):
title: str = Field(description="Short title for the ticket")
description: str = Field(description="Detailed description of the issue")
priority: Priority = Field(description="Priority level of the ticket")
category: TicketCategory = Field(description="Category of the ticket")
tags: List[str] = Field(default_factory=list, description="Relevant tags")
estimated_resolution_hours: Optional[int] = Field(
default=None, description="Estimated hours to resolve", ge=1
)
agent = Agent(
"openai:gpt-4o",
output_type=SupportTicket,
system_prompt=(
"You are a support ticket triage assistant. Analyze the user's "
"message and create a structured support ticket with appropriate "
"priority, category, and tags."
),
)
result = agent.run_sync(
"I've been charged twice for my subscription this month and I can't "
"access my account. This is urgent, I need this fixed immediately!"
)
ticket = result.output
print(f"Title: {ticket.title}")
print(f"Priority: {ticket.priority.value}")
print(f"Category: {ticket.category.value}")
print(f"Tags: {ticket.tags}")
print(f"Est. resolution: {ticket.estimated_resolution_hours} hours")Using str, Enum ensures the serialized values are human-readable strings, which helps the LLM understand the expected format. If the model returns "urgent" instead of one of the defined priorities, validation fails and the model is asked to correct itself.
Working with Tool Calls and Structured Outputs
Pydantic AI also applies structured validation to tool definitions. When you define a tool, its arguments are validated using Pydantic models, ensuring the LLM calls tools with the correct parameters.
from pydantic import BaseModel, Field
from pydantic_ai import Agent, RunContext
class WeatherRequest(BaseModel):
city: str = Field(description="The city to get weather for")
country: str = Field(default="US", description="Country code")
units: str = Field(default="celsius", description="Temperature units")
agent = Agent("openai:gpt-4o", system_prompt="You are a weather assistant.")
@agent.tool
async def get_weather(ctx: RunContext[None], params: WeatherRequest) -> dict:
"""Get the current weather for a city."""
# In a real app, you'd call a weather API here
# The params object is already validated by Pydantic
return {
"city": params.city,
"country": params.country,
"temperature": 22,
"units": params.units,
"condition": "sunny",
}
result = agent.run_sync("What's the weather like in Paris, France?")
print(result.output)The WeatherRequest model validates the arguments the LLM passes to the tool. If the model omits the city field or passes an invalid type, Pydantic catches it before your tool function ever executes.
Streaming with Structured Outputs
Pydantic AI supports streaming while still providing structured output validation. The framework incrementally parses the streamed JSON and validates it once complete.
import asyncio
from pydantic import BaseModel, Field
from pydantic_ai import Agent
class StorySummary(BaseModel):
title: str = Field(description="A catchy title for the story")
genre: str = Field(description="The literary genre")
word_count: int = Field(description="Approximate word count", ge=1)
themes: list[str] = Field(description="Key themes in the story")
summary: str = Field(description="A one-paragraph summary")
agent = Agent(
"openai:gpt-4o",
output_type=StorySummary,
system_prompt="You are a literary analyst. Analyze stories and return structured summaries.",
)
async def main():
story_text = (
"The old lighthouse keeper had seen many storms, but none like this. "
"As waves crashed against the rocks, he spotted a small boat struggling "
"in the distance. What followed was a night of courage, sacrifice, and "
"the discovery that some lights shine brightest in the darkest hours."
)
async with agent.run_stream(story_text) as result:
# Stream partial outputs as they arrive
async for partial in result.stream():
if partial is not None:
print(f"Partial title: {partial.title}")
# Get the final validated output
final = await result.get_output()
print(f"\nFinal result:")
print(final.model_dump_json(indent=2))
asyncio.run(main())During streaming, partial results may be None until enough data has arrived to form a valid partial object. The final get_output() call returns the fully validated model.
Best Practices for Structured Output Validation
1. Write Descriptive Field Descriptions
Field descriptions are your primary mechanism for guiding the LLM. Be specific about format, allowed values, and intent. Vague descriptions lead to inconsistent outputs.
# Poor
class Bad(BaseModel):
name: str
value: float
# Good
class Good(BaseModel):
name: str = Field(description="Full legal name in 'First Last' format")
value: float = Field(
description="Monetary value in USD, rounded to 2 decimal places",
ge=0
)2. Use Constraints Liberally
Pydantic's constraint features (ge, le, min_length, max_length, pattern) prevent a wide class of errors. The more constraints you define, the more the retry mechanism can help the LLM self-correct.
from pydantic import BaseModel, Field
class Order(BaseModel):
order_id: str = Field(
pattern=r"^ORD-\d{6}$",
description="Order ID in format ORD-XXXXXX where X is a digit"
)
quantity: int = Field(ge=1, le=9999, description="Number of items ordered")
price_per_unit: float = Field(gt=0, description="Price per unit in USD")
discount_percent: float = Field(
default=0.0, ge=0, le=100, description="Discount percentage 0-100"
)3. Keep Models Focused
Avoid creating massive models with dozens of fields. Large schemas are harder for LLMs to fill correctly. If you need complex output, break it into smaller, composable models or use multiple agent calls.
4. Set Reasonable Retry Limits
The retries parameter controls how many times Pydantic AI will retry after a validation failure. The default is 1. For complex schemas, increase it to 3 or 4. For simple schemas, 1 or 2 is usually sufficient. Setting it too high wastes tokens; too low risks failures on edge cases.
5. Use model_dump() for Serialization
When you need to convert your validated output to a dictionary or JSON for storage or API responses, use Pydantic's built-in serialization methods rather than manual conversion.
customer = result.output
# To dictionary
data = customer.model_dump()
# To JSON string
json_str = customer.model_dump_json(indent=2)
# Exclude certain fields
public_data = customer.model_dump(exclude={"is_vip"})6. Handle Edge Cases with Optional Fields and Defaults
Not all information will always be present in the input. Use Optional types and sensible defaults so the model is not forced to hallucinate values.
from typing import Optional
from pydantic import BaseModel, Field
class ContactInfo(BaseModel):
name: str = Field(description="Person's full name")
email: Optional[str] = Field(default=None, description="Email if available")
phone: Optional[str] = Field(default=None, description="Phone if available")
preferred_contact: str = Field(
default="email", description="Preferred contact method: email or phone"
)7. Test Your Models Independently
Your Pydantic models are plain Python classes. You can and should unit test them independently of the LLM to ensure your validation logic is correct.
import pytest
from pydantic import ValidationError
def test_valid_customer():
customer = CustomerInfo(
name="Jane Doe", age=34, email="jane@example.com", is_vip=True
)
assert customer.name == "Jane Doe"
assert customer.age == 34
def test_invalid_age_raises():
with pytest.raises(ValidationError):
CustomerInfo(
name="Jane Doe", age=200, email="jane@example.com"
)
def test_invalid_email_raises():
with pytest.raises(ValidationError):
CustomerInfo(
name="Jane Doe", age=34, email="not-an-email"
)Debugging Validation Failures
When validation fails after all retries are exhausted, Pydantic AI raises a ValidationError or an UnexpectedModelBehavior exception. You can inspect these to understand what went wrong.
from pydantic_ai import Agent
from pydantic_ai.exceptions import UnexpectedModelBehavior
from pydantic import BaseModel, Field
class StrictModel(BaseModel):
code: str = Field(pattern=r"^[A-Z]{3}-\d{4}$", description="Code in format ABC-1234")
agent = Agent("openai:gpt-4o", output_type=StrictModel, retries=2)
try:
result = agent.run_sync("Generate a product code for a widget.")
print(result.output)
except UnexpectedModelBehavior as e:
print(f"Model failed after retries: {e}")
# Inspect the message history to see what the model produced
# and what validation errors occurred
except Exception as e:
print(f"Other error: {e}")You can also enable debug logging to see the full interaction between Pydantic AI and the LLM, including the validation errors that were sent back during retries.
import logging
logging.basicConfig(level=logging.DEBUG)
logging.getLogger("pydantic_ai").setLevel(logging.DEBUG)Conclusion
Structured output validation with Pydantic AI transforms LLM outputs from unpredictable text into reliable, typed Python objects. By defining clear Pydantic models with descriptive fields, meaningful constraints, and custom validators, you give the LLM a precise target and give your application a safety net. The automatic retry mechanism turns validation failures from hard crashes into self-correcting loops, dramatically improving reliability. Whether you are building simple extraction tasks or complex multi-step agent pipelines, making structured validation a first-class concern will save you from countless bugs, reduce defensive coding, and let you focus on your application's actual business logic. Start with focused models, write clear descriptions, test your validators independently, and let Pydantic AI handle the hard work of keeping your LLM outputs trustworthy.