Introduction to Structured Tool Calling
Structured tool calling is one of the most powerful capabilities in modern LLM development. Instead of relying on the model to output free-form text that you then parse with fragile regular expressions, you can instruct the model to return a structured JSON object that conforms to a schema you define. The OpenAI Function Calling API makes this possible by letting you declare "tools" (functions) with typed parameters, and the model decides when and how to call them.
This tutorial walks through everything you need to know: what function calling is, why it matters, how to implement it end-to-end, and the best practices that separate production-grade integrations from toy demos.
What Is the OpenAI Function Calling API?
The Function Calling API allows you to describe functions to the model using a JSON schema. When the model determines that calling a function would help answer the user's request, it returns a structured response containing the function name and arguments as valid JSON. Your application code then executes the actual function and feeds the result back to the model in a subsequent turn.
The key insight is that the model does not execute the function itself. It only produces the structured arguments. Your code is responsible for the actual execution, which gives you full control over safety, validation, and side effects.
Core Concepts
- Tools: The array of function definitions you pass to the model, each with a name, description, and parameter schema.
- Tool calls: The structured output the model returns when it wants to invoke one of your functions.
- Tool messages: The messages you send back to the model containing the results of executed function calls.
- Parallel tool calls: The model can request multiple function calls in a single response, which you can execute concurrently.
Why Structured Tool Calling Matters
Before function calling existed, developers tried to coax models into producing parseable output through prompt engineering. This approach was brittle: models would add commentary, wrap JSON in markdown fences, or invent fields. Structured tool calling solves several real problems:
- Reliability: The model is trained to produce JSON conforming to your schema, dramatically reducing parsing failures.
- Type safety: You declare parameter types, enums, and required fields, so the model knows exactly what shape to produce.
- Composability: You can expose multiple tools and let the model choose the right one, enabling agentic workflows.
- Auditability: Every tool call is an explicit, inspectable event in your conversation history.
- Security boundary: The model never executes code directly; your application acts as the gatekeeper.
How to Use the Function Calling API
1. Defining Your Tools
Each tool is defined as a JSON object with a type of "function" and a nested function definition. The parameter schema follows JSON Schema conventions.
tools = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the current weather for a given city. Use this when the user asks about temperature, conditions, or forecasts.",
"parameters": {
"type": "object",
"properties": {
"city": {
"type": "string",
"description": "The name of the city, e.g. 'San Francisco'"
},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "Temperature unit to return"
}
},
"required": ["city"],
"additionalProperties": False
}
}
},
{
"type": "function",
"function": {
"name": "search_products",
"description": "Search the product catalog by keyword and optional category filter.",
"parameters": {
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "Search keywords"
},
"category": {
"type": "string",
"enum": ["electronics", "books", "clothing", "home"],
"description": "Optional category filter"
},
"max_results": {
"type": "integer",
"description": "Maximum number of results to return",
"default": 10
}
},
"required": ["query"],
"additionalProperties": False
}
}
}
]
Notice how each property has a description. These descriptions are part of the prompt context the model sees, so clear, specific descriptions directly improve call accuracy.
2. Making the Initial Request
Pass the tools array along with your messages to the chat completions endpoint. You can also control tool selection behavior with the tool_choice parameter.
from openai import OpenAI
client = OpenAI()
messages = [
{
"role": "user",
"content": "What's the weather in Tokyo right now in celsius?"
}
]
response = client.chat.completions.create(
model="gpt-4o",
messages=messages,
tools=tools,
tool_choice="auto"
)
message = response.choices[0].message
print(message.tool_calls)
The tool_choice parameter accepts several values:
"auto"— the model decides whether to call a tool (default)."none"— the model will not call any tools."required"— the model must call at least one tool.{"type": "function", "function": {"name": "get_weather"}}— force a specific function.
3. Handling the Tool Call Response
When the model decides to call a function, the response message will contain a tool_calls array. Each entry has an ID, a function name, and stringified JSON arguments.
import json
if message.tool_calls:
# Append the assistant message with tool calls to the conversation
messages.append(message)
for tool_call in message.tool_calls:
function_name = tool_call.function.name
arguments = json.loads(tool_call.function.arguments)
print(f"Model wants to call: {function_name}")
print(f"With arguments: {arguments}")
# Dispatch to the actual function
if function_name == "get_weather":
result = get_weather(**arguments)
elif function_name == "search_products":
result = search_products(**arguments)
else:
result = {"error": f"Unknown function: {function_name}"}
# Send the result back to the model
messages.append({
"role": "tool",
"tool_call_id": tool_call.id,
"content": json.dumps(result)
})
4. Implementing the Actual Functions
Your functions are regular Python functions. They can call external APIs, query databases, or perform any computation. Here is a simple mock implementation:
def get_weather(city: str, unit: str = "celsius") -> dict:
"""In production, call a real weather API here."""
# Simulated response
return {
"city": city,
"temperature": 22 if unit == "celsius" else 72,
"unit": unit,
"condition": "Partly cloudy",
"humidity": 65
}
def search_products(query: str, category: str = None, max_results: int = 10) -> dict:
"""In production, query your product database."""
# Simulated response
results = [
{"id": 1, "name": f"{query} item A", "price": 29.99},
{"id": 2, "name": f"{query} item B", "price": 49.99},
]
if category:
results = [r for r in results if category in r["name"].lower()]
return {
"query": query,
"category": category,
"count": len(results[:max_results]),
"results": results[:max_results]
}
5. Completing the Conversation Loop
After sending the tool results back, make a second request so the model can synthesize a natural-language response using the function output.
final_response = client.chat.completions.create(
model="gpt-4o",
messages=messages,
tools=tools,
tool_choice="auto"
)
print(final_response.choices[0].message.content)
# Output: "The current weather in Tokyo is 22°C with partly cloudy skies and 65% humidity."
6. Putting It All Together: A Full Agent Loop
In real applications, the model may need to call multiple tools in sequence. A robust implementation wraps the entire flow in a loop that continues until the model stops requesting tool calls.
import json
from openai import OpenAI
client = OpenAI()
def run_agent(user_input: str, tools: list, max_iterations: int = 10) -> str:
messages = [{"role": "user", "content": user_input}]
# Map function names to callables
function_map = {
"get_weather": get_weather,
"search_products": search_products,
}
for _ in range(max_iterations):
response = client.chat.completions.create(
model="gpt-4o",
messages=messages,
tools=tools,
tool_choice="auto"
)
msg = response.choices[0].message
messages.append(msg)
# If no tool calls, the model is done — return its text response
if not msg.tool_calls:
return msg.content
# Execute each requested tool call
for tool_call in msg.tool_calls:
name = tool_call.function.name
args = json.loads(tool_call.function.arguments)
try:
func = function_map.get(name)
if func is None:
result = {"error": f"Unknown function: {name}"}
else:
result = func(**args)
except Exception as e:
result = {"error": str(e)}
messages.append({
"role": "tool",
"tool_call_id": tool_call.id,
"content": json.dumps(result)
})
return "Reached maximum iterations without a final answer."
You can now run multi-step queries like this:
answer = run_agent(
"I'm planning a trip to Tokyo. What's the weather like, "
"and can you search for travel books?",
tools=tools
)
print(answer)
The model will first call get_weather, then search_products, and finally compose a coherent answer using both results.
Best Practices
Write Detailed Function and Parameter Descriptions
The descriptions you write are the model's only guidance for when and how to use each tool. Vague descriptions lead to missed calls or incorrect arguments. Be explicit about purpose, expected formats, and edge cases. For example, instead of "The city", write "The full name of the city including country if ambiguous, e.g. 'Portland, OR' or 'Portland, ME'".
Validate Arguments Before Execution
Never trust the model's output blindly. Even with structured calling, the model can occasionally produce unexpected values. Use a validation library like Pydantic to enforce types and constraints before executing the function.
from pydantic import BaseModel, ValidationError
from typing import Optional, Literal
class WeatherArgs(BaseModel):
city: str
unit: Literal["celsius", "fahrenheit"] = "celsius"
def safe_get_weather(raw_args: dict) -> dict:
try:
args = WeatherArgs(**raw_args)
return get_weather(**args.model_dump())
except ValidationError as e:
return {"error": "Invalid arguments", "details": e.errors()}
Handle Errors Gracefully and Feed Them Back
When a function fails, return the error as a structured tool message rather than crashing. The model can often recover by adjusting its arguments or trying a different approach.
try:
result = func(**args)
except Exception as e:
result = {
"error": True,
"message": f"Function '{name}' failed: {str(e)}",
"suggestion": "Check the argument types and try again."
}
Limit the Number of Tools
While you can pass many tools, performance and accuracy degrade when the list grows too large. If you have dozens of functions, consider grouping them by domain and dynamically selecting a relevant subset based on the user's query, or use a retrieval step to find the most applicable tools.
Use strict Mode for Guaranteed Schema Compliance
OpenAI supports a strict flag on function definitions that guarantees the model's output will match your schema exactly, including no extra properties and no missing required fields. This is strongly recommended for production use.
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get current weather for a city.",
"strict": True,
"parameters": {
"type": "object",
"properties": {
"city": {"type": "string"},
"unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
},
"required": ["city", "unit"],
"additionalProperties": False
}
}
}
Note that when using strict mode, all parameters must be listed in required, even optional ones. You handle optionality by allowing null values in the type definition.
Log Every Tool Call
For debugging and observability, log each tool call with its arguments, the result, and the latency. This creates an audit trail and helps you identify cases where the model misuses a function.
import logging
import time
logger = logging.getLogger("tool_calls")
for tool_call in msg.tool_calls:
name = tool_call.function.name
args = json.loads(tool_call.function.arguments)
start = time.time()
result = function_map[name](**args)
elapsed = time.time() - start
logger.info({
"function": name,
"arguments": args,
"latency_ms": round(elapsed * 1000, 2),
"success": "error" not in result if isinstance(result, dict) else True
})
Cap Iterations to Prevent Infinite Loops
As shown in the agent loop example, always set a maximum iteration count. Without it, a model that repeatedly calls tools without converging on an answer can generate unbounded API costs.
Conclusion
Structured tool calling with the OpenAI Function Calling API transforms LLMs from text generators into reliable components of larger systems. By defining clear schemas, validating inputs, handling errors gracefully, and following best practices like strict mode and iteration caps, you can build robust agents that interact with real-world APIs and databases safely and predictably. The pattern is straightforward — define your tools, let the model decide when to call them, execute them in your own code, and feed the results back — but the discipline you apply around that loop is what makes the difference between a fragile prototype and a production-ready application. Start simple with one or two well-described functions, then expand your toolset as you gain confidence in the model's ability to select and parameterize them correctly.