Introduction to Tool Use Patterns with LlamaIndex
Tool use is one of the most powerful capabilities in modern LLM applications. Instead of relying solely on a model's parametric knowledge, you can equip agents with external tools — functions, APIs, query engines, and retrievers — that allow them to take real actions and access fresh information. LlamaIndex provides a rich, composable framework for defining, registering, and orchestrating these tools.
In this guide, we'll explore the major tool use patterns supported by LlamaIndex, from simple function tools to multi-agent orchestration. Each pattern comes with a runnable code example and a discussion of when to apply it.
What Is Tool Use in LlamaIndex?
At its core, tool use means giving an LLM the ability to call external capabilities during a conversation. LlamaIndex abstracts this through the BaseTool interface. A tool wraps some callable — a Python function, a query engine, a retriever, or another agent — and exposes a schema the LLM can reason about.
When a tool is attached to an agent (such as FunctionAgent or ReActAgent), the agent decides at inference time which tool to invoke, with what arguments, and how to incorporate the result into its response. This loop — plan, call, observe, respond — is the foundation of agentic behavior.
Core Building Blocks
FunctionTool— wraps any Python function and infers its schema from the signature and docstring.QueryEngineTool— wraps a LlamaIndex query engine (e.g., RAG over documents).RetrieverTool— wraps a retriever for direct document fetching.ToolSpec/ToolCollections— group multiple tools together for reuse.AgentWorker/AgentRunner— orchestrate tool calls within an agent loop.
Why Tool Use Matters
LLMs alone have well-known limitations: their training data has a cutoff, they cannot perform precise arithmetic reliably, and they cannot interact with external systems. Tools address all three problems:
- Freshness: Query engines and API tools retrieve up-to-date information.
- Accuracy: Calculator, SQL, and code-execution tools produce deterministic results.
- Action: Tools can send emails, update databases, or trigger workflows.
- Modularity: Complex capabilities can be composed from small, testable units.
- Cost control: Routing to specialized tools often reduces token usage compared to stuffing context.
Choosing the right tool use pattern is therefore as important as choosing the model itself. Let's walk through the major patterns.
Pattern 1: Simple Function Tools
The simplest pattern wraps a Python function as a tool and hands it to a ReAct-style agent. LlamaIndex inspects the function signature and docstring to generate the JSON schema the model uses for tool calling.
from llama_index.core.agent import FunctionAgent
from llama_index.core.tools import FunctionTool
from llama_index.llms.openai import OpenAI
def add(a: float, b: float) -> float:
"""Add two numbers and return the result."""
return a + b
def multiply(a: float, b: float) -> float:
"""Multiply two numbers and return the result."""
return a * b
add_tool = FunctionTool.from_defaults(fn=add)
multiply_tool = FunctionTool.from_defaults(fn=multiply)
llm = OpenAI(model="gpt-4o-mini")
agent = FunctionAgent(
tools=[add_tool, multiply_tool],
llm=llm,
system_prompt="You are a helpful math assistant. Use tools for calculations.",
)
response = agent.run("What is (12 + 7) * 3?")
print(response)
Here the agent will call add(12, 7), observe 19, then call multiply(19, 3), and finally return 57. The docstrings are critical — they are the model's only signal for when to use each tool.
When to Use This Pattern
- You have a handful of well-defined utility functions.
- The logic is deterministic and best expressed in code.
- You want minimal infrastructure and fast iteration.
Pattern 2: Query Engine Tools (RAG)
When your tool needs to answer questions over a corpus of documents, wrap a query engine with QueryEngineTool. This is the canonical RAG-as-a-tool pattern: the agent decides when retrieval is necessary and what question to ask.
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
from llama_index.core.tools import QueryEngineTool, ToolMetadata
from llama_index.core.agent import FunctionAgent
from llama_index.llms.openai import OpenAI
documents = SimpleDirectoryReader("./data/policies").load_data()
index = VectorStoreIndex.from_documents(documents)
policy_engine = index.as_query_engine(similarity_top_k=3)
policy_tool = QueryEngineTool(
query_engine=policy_engine,
metadata=ToolMetadata(
name="policy_search",
description=(
"Useful for answering questions about company HR policies, "
"vacation rules, and benefits. Input should be a natural language question."
),
),
)
llm = OpenAI(model="gpt-4o-mini")
agent = FunctionAgent(tools=[policy_tool], llm=llm)
response = agent.run("How many vacation days do new employees get in their first year?")
print(response)
Note the description field — it tells the agent both what the tool does and how to format its input. A vague description is the most common cause of poor tool selection.
Pattern 3: Multi-Tool Routing
Real applications rarely have a single tool. The multi-tool pattern gives the agent several tools and lets it route between them based on the user's question. This is the foundation of "agent-as-router" architectures.
from llama_index.core.tools import FunctionTool, QueryEngineTool, ToolMetadata
from llama_index.core.agent import FunctionAgent
from llama_index.llms.openai import OpenAI
# Tool 1: live weather stub
def get_weather(city: str) -> str:
"""Return the current weather for a city."""
# In production, call a real API here.
return f"The weather in {city} is sunny and 22C."
weather_tool = FunctionTool.from_defaults(fn=get_weather)
# Tool 2: knowledge base
# (assume kb_engine is a QueryEngine built earlier)
kb_tool = QueryEngineTool(
query_engine=kb_engine,
metadata=ToolMetadata(
name="kb_search",
description="Search the internal knowledge base for product documentation.",
),
)
# Tool 3: calculator
def calculate(expression: str) -> str:
"""Evaluate a math expression like '3 * (4 + 2)'."""
try:
return str(eval(expression, {"__builtins__": {}}, {}))
except Exception as e:
return f"Error: {e}"
calc_tool = FunctionTool.from_defaults(fn=calculate)
llm = OpenAI(model="gpt-4o-mini")
agent = FunctionAgent(
tools=[weather_tool, kb_tool, calc_tool],
llm=llm,
system_prompt=(
"You are a helpful assistant. Pick the right tool for each question. "
"If no tool fits, answer from general knowledge."
),
)
print(agent.run("What's the weather in Tokyo and what is 18 * 24?"))
The agent will typically call both get_weather and calculate in a single turn and synthesize the results. This implicit parallelism is a major productivity win over hand-coded routing.
Pattern 4: Sub-Question Decomposition
Sometimes a single user question requires querying multiple sources or asking multiple sub-questions. The SubQuestionQueryEngine decomposes a complex query into focused sub-queries, each routed to the appropriate tool, then synthesizes the answers.
from llama_index.core.tools import QueryEngineTool, ToolMetadata
from llama_index.core.query_engine import SubQuestionQueryEngine
from llama_index.core import VectorStoreIndex
# Two separate indexes: one for 2023 reports, one for 2024 reports
reports_2023 = VectorStoreIndex.from_documents(docs_2023).as_query_engine()
reports_2024 = VectorStoreIndex.from_documents(docs_2024).as_query_engine()
tool_2023 = QueryEngineTool.from_defaults(
query_engine=reports_2023,
name="reports_2023",
description="Financial reports from 2023.",
)
tool_2024 = QueryEngineTool.from_defaults(
query_engine=reports_2024,
name="reports_2024",
description="Financial reports from 2024.",
)
sub_engine = SubQuestionQueryEngine.from_defaults(
query_engine_tools=[tool_2023, tool_2024]
)
response = sub_engine.query(
"Compare revenue growth between 2023 and 2024."
)
print(response)
Under the hood, the engine generates sub-questions like "What was revenue in 2023?" and "What was revenue in 2024?", dispatches each to the matching tool, and merges the results. This pattern shines for comparative and multi-source analysis.
Pattern 5: Agent-as-Tool (Nested Agents)
For complex systems, you can wrap an entire agent as a tool that another agent calls. This enables hierarchical orchestration: a top-level "orchestrator" agent delegates to specialized sub-agents.
from llama_index.core.agent import FunctionAgent
from llama_index.core.tools import FunctionTool
from llama_index.llms.openai import OpenAI
llm = OpenAI(model="gpt-4o-mini")
# Research sub-agent
research_agent = FunctionAgent(
tools=[web_search_tool, kb_tool],
llm=llm,
system_prompt="You are a research specialist. Gather information and summarize.",
)
# Wrap the sub-agent as a callable tool
def research(topic: str) -> str:
"""Research a topic using web and knowledge base tools."""
return str(research_agent.run(topic))
research_tool = FunctionTool.from_defaults(fn=research)
# Writer sub-agent
writer_agent = FunctionAgent(
tools=[],
llm=llm,
system_prompt="You are a technical writer. Produce clean prose.",
)
def write_draft(notes: str) -> str:
"""Turn research notes into a polished article draft."""
return str(writer_agent.run(f"Write an article based on these notes:\n{notes}"))
write_tool = FunctionTool.from_defaults(fn=write_draft)
# Orchestrator
orchestrator = FunctionAgent(
tools=[research_tool, write_tool],
llm=llm,
system_prompt=(
"You coordinate a research-and-writing pipeline. "
"First research the topic, then write a draft from the findings."
),
)
print(orchestrator.run("Write a short article about retrieval-augmented generation."))
This pattern keeps each agent's prompt and toolset small, which improves reliability. It also mirrors how human teams work: specialists handle depth, a coordinator handles flow.
Pattern 6: Custom Tools with Async and Context
Production tools often need async I/O, shared state, or dependency injection. LlamaIndex supports async tool functions and lets you close over context via factories or classes.
import asyncio
from llama_index.core.tools import FunctionTool
from llama_index.core.agent import FunctionAgent
from llama_index.llms.openai import OpenAI
class DatabaseSession:
def __init__(self, dsn: str):
self.dsn = dsn
async def fetch_user(self, user_id: int) -> dict:
# Simulate async DB call
await asyncio.sleep(0.1)
return {"id": user_id, "name": "Ada Lovelace", "plan": "pro"}
session = DatabaseSession(dsn="postgres://localhost/app")
async def get_user_profile(user_id: int) -> str:
"""Fetch a user's profile by ID. Returns a short summary string."""
user = await session.fetch_user(user_id)
return f"User {user['name']} (id={user['id']}) is on the {user['plan']} plan."
user_tool = FunctionTool.from_defaults(fn=get_user_profile, async_mode=True)
llm = OpenAI(model="gpt-4o-mini")
agent = FunctionAgent(tools=[user_tool], llm=llm)
result = asyncio.run(agent.run("Show me the profile for user 42."))
print(result)
Using a class to hold the session means the tool function stays pure and testable — you can swap the session in tests without touching the agent code.
Pattern 7: Constrained Tool Output with Pydantic
When a tool's output must conform to a strict schema (for downstream parsing or API calls), use Pydantic models. LlamaIndex integrates cleanly with structured outputs.
from pydantic import BaseModel, Field
from llama_index.core.tools import FunctionTool
class InvoiceSummary(BaseModel):
total: float = Field(description="Total invoice amount in USD")
line_items: int = Field(description="Number of line items")
overdue: bool = Field(description="Whether the invoice is overdue")
def summarize_invoice(invoice_text: str) -> InvoiceSummary:
"""Parse invoice text and return a structured summary."""
# In practice, call an LLM with structured output or a parser.
return InvoiceSummary(total=1240.50, line_items=3, overdue=False)
invoice_tool = FunctionTool.from_defaults(fn=summarize_invoice)
Structured tool outputs make it safe to chain tools: one tool's output becomes another's typed input, reducing the chance of malformed intermediate data.
Best Practices
Write Excellent Tool Descriptions
The description is the single biggest lever for tool selection quality. Be specific about scope, input format, and limitations. Bad: "Searches things." Good: "Searches the product documentation index. Input should be a concise natural language question about product features."
Keep Toolsets Small and Focused
Models degrade at selecting among dozens of similar tools. Aim for 3–8 tools per agent. If you need more, split into sub-agents (Pattern 5) or group related tools with metadata prefixes.
Make Tools Idempotent Where Possible
Side-effecting tools (writes, sends, deletes) should be idempotent or guarded by confirmation prompts. Agents sometimes retry calls; idempotency prevents duplicate side effects.
Validate Inputs at the Boundary
Even though the LLM produces arguments, always validate them inside the tool. Type hints help, but explicit checks (ranges, enums, permissions) protect against hallucinated arguments.
Log Every Tool Call
Instrument tools with logging or callbacks. LlamaIndex's callback manager lets you trace each call, its arguments, and its return value — invaluable for debugging agent behavior.
from llama_index.core import Settings
from llama_index.core.callbacks import CallbackManager, LlamaDebugHandler
debug_handler = LlamaDebugHandler(print_trace_on_end=True)
Settings.callback_manager = CallbackManager([debug_handler])
# ... build and run your agent ...
# After running, inspect:
for event in debug_handler.get_events():
print(event)
Handle Tool Failures Gracefully
Return descriptive error strings from tools rather than raising exceptions. The agent can then reason about the failure and try an alternative. For example, return "Error: city not found. Try a major city name." instead of crashing.
Test Tools Independently
Because tools are plain Python functions, you can unit-test them without the LLM. This dramatically shortens the debug loop. Only integration-test the full agent once tools are solid.
Conclusion
Tool use transforms LLMs from text generators into capable agents that retrieve, compute, and act. LlamaIndex's tool abstractions — FunctionTool, QueryEngineTool, SubQuestionQueryEngine, and nested agents — give you a graduated set of patterns from simple function calls to hierarchical multi-agent systems. The key to success is not the framework itself but the discipline of writing precise descriptions, keeping toolsets focused, validating inputs, and logging every call. Start with Pattern 1, add tools as real needs emerge, and reach for decomposition or nested agents only when a single agent's toolset grows unwieldy. With these patterns and practices, you can build LLM applications that are reliable, observable, and genuinely useful in production.