Function Calling at Scale with Pydantic AI: Complete Guide

Introduction to Function Calling at Scale with Pydantic AI

Function calling has transformed how developers build applications with Large Language Models (LLMs). Instead of merely generating text, LLMs can now invoke external tools, query databases, and interact with APIs to fetch real-time data or execute actions. However, when you need to implement function calling at scale—managing dozens of tools, handling complex dependencies, and ensuring high reliability—the standard LLM APIs can quickly become brittle and difficult to maintain.

Pydantic AI is a powerful framework designed to bridge the gap between LLMs and structured Python programming. By leveraging Pydantic's robust type validation, Pydantic AI allows developers to define tools using standard Python type hints, automatically generating the JSON schemas required by LLMs. This approach drastically reduces boilerplate and ensures that the data passed between the LLM and your application is strictly validated.

Why Pydantic AI for Function Calling?

When building production-grade AI applications, unstructured outputs and hallucinated function arguments are major pain points. Pydantic AI addresses these challenges by making type safety the core of the agent framework. Instead of manually writing JSON schemas for your functions, you write standard Python functions, and Pydantic AI handles the rest.

Automatic Schema Generation: Your Python type hints are automatically converted into the JSON schema required by the LLM.
Type Safety and Validation: Inputs generated by the LLM are validated against your Pydantic models before your function is ever executed.
Dependency Injection: Pydantic AI provides a clean way to pass database connections, API clients, and other stateful objects directly to your tools.
Structured Outputs: You can enforce a specific Pydantic model as the final output of the agent, guaranteeing that your application receives data in the exact format it expects.

Getting Started: Basic Function Calling

To begin, you need to install the Pydantic AI package. You can do this via pip. Ensure you also have your LLM provider's API key set in your environment variables.

pip install pydantic-ai

Let's start with a simple example. We will create an agent that has access to a tool for fetching a user's account balance. Notice how we use standard Python type hints for the arguments.

from pydantic_ai import Agent

# Initialize the agent with the desired LLM model
agent = Agent('openai:gpt-4o')

# Define a tool using the @agent.tool decorator
@agent.tool
def get_account_balance(ctx, account_id: str) -> str:
    """Fetch the current balance for a given account ID."""
    # In a real application, this would be a database query
    mock_balances = {
        "12345": "$1,250.00",
        "67890": "$50.75"
    }
    return mock_balances.get(account_id, "Account not found.")

# Run the agent with a user prompt
result = agent.run_sync("What is the balance for account 12345?")
print(result.data)
# Output: The balance for account 12345 is $1,250.00.

In this example, the LLM reads the docstring and the account_id: str type hint to understand how to use the get_account_balance tool. When the user asks about the balance, the LLM generates the function call, Pydantic AI executes it, and the result is fed back to the LLM to generate the final response.

Scaling Up: Managing Multiple Tools and Dependencies

When scaling up, your tools will likely need access to external resources like databases or third-party APIs. Hardcoding these connections inside your tool functions is an anti-pattern. Pydantic AI solves this using a powerful dependency injection system.

Implementing Dependency Injection

You can define a dataclass or Pydantic model to hold your dependencies, pass it to the agent at runtime, and access it via the ctx (context) object inside your tools.

from dataclasses import dataclass
from pydantic_ai import Agent

# Define a dependency container
@dataclass
class DatabaseClient:
    async def fetch_user_data(self, user_id: str) -> dict:
        # Simulating an async database fetch
        return {"user_id": user_id, "name": "Alice", "status": "active"}

# Initialize the agent, specifying the dependency type
agent = Agent('openai:gpt-4o', deps_type=DatabaseClient)

# Define an async tool that uses the injected dependency
@agent.tool
async def fetch_user_profile(ctx, user_id: str) -> dict:
    """Fetch a user's profile information."""
    db_client = ctx.deps
    user_data = await db_client.fetch_user_data(user_id)
    return user_data

# Instantiate your dependency
db = DatabaseClient()

# Run the agent, passing the dependency instance
result = agent.run_sync("Get the profile for user 999.", deps=db)
print(result.data)

This pattern allows you to easily mock dependencies for testing, share connection pools across multiple tools, and keep your business logic cleanly separated from infrastructure concerns.

Best Practices for Function Calling at Scale

As your application grows and you expose dozens of tools to the LLM, following best practices becomes critical for performance and reliability.

Keep Tools Atomic: Each tool should do one thing and do it well. Instead of a single manage_user tool that creates, updates, and deletes, create three separate tools: create_user, update_user, and delete_user. This makes it easier for the LLM to select the correct tool.
Write Clear Docstrings: The LLM relies heavily on your docstrings to understand when and how to use a tool. Be explicit about what the tool does, what the arguments mean, and what it returns.
Use Strict Pydantic Models: For complex inputs, define a Pydantic BaseModel instead of using multiple primitive arguments. This provides an extra layer of validation and makes the schema cleaner for the LLM.
Handle Errors Gracefully: If a tool fails (e.g., a database timeout), catch the exception and return a structured error message to the LLM rather than raising an exception. This allows the LLM to inform the user or try an alternative approach.
Limit Tool Scope per Agent: If you have 50 tools, consider breaking your application into multiple specialized agents, each with a smaller subset of tools. This reduces context window usage and improves the LLM's tool selection accuracy.

Conclusion

Function calling is the cornerstone of building agentic, action-oriented AI applications, but managing it at scale requires a structured approach. Pydantic AI provides an elegant solution by combining the flexibility of LLMs with the rigorous type safety of Pydantic. By leveraging automatic schema generation, robust dependency injection, and strict validation, developers can build complex, multi-tool agents that are reliable, maintainable, and easy to test. By following atomic design principles and keeping your tool definitions clean, you can confidently scale your AI applications from simple prototypes to robust production systems.