How to Handle Tool Execution Failures in LLM Agents

LLM agents are powerful because they can call external tools — search APIs, databases, code interpreters, file systems — to extend their reasoning with real-world actions. But every tool call is a potential point of failure. APIs rate-limit you, network connections drop, JSON payloads are malformed, and permission errors lurk behind every corner. If your agent doesn't handle these failures gracefully, a single transient error can derail an entire task, waste tokens, or worse, cause the agent to hallucinate a result it never actually obtained.

This tutorial walks through what tool execution failures are, why they matter, and how to build robust failure-handling strategies into your LLM agent loop. We'll use Python for the examples, but the patterns apply to any language.

What Is a Tool Execution Failure?

A tool execution failure is any condition where the agent's attempt to invoke a tool does not produce the expected successful result. Failures generally fall into four categories:

Transient errors: Network timeouts, rate limits (HTTP 429), temporary DNS failures. These often succeed on retry.
Validation errors: The LLM produced arguments that don't match the tool's schema — wrong types, missing required fields, out-of-range values.
Permission or auth errors: Expired tokens, insufficient scopes, forbidden resources (HTTP 401/403).
Logical errors: The tool ran successfully but returned an error semantic to the agent — e.g., a search returned zero results, or a database query found no matching rows.

Each category requires a different response strategy. Treating them all the same — for example, blindly retrying a validation error — wastes resources and confuses the model.

Why Failure Handling Matters

Without explicit failure handling, your agent will likely do one of two things, both bad:

Crash: An unhandled exception terminates the agent loop, and the user sees a stack trace instead of a useful answer.
Hallucinate: The agent receives a vague error string, misinterprets it as a successful (but empty) result, and fabricates a plausible-sounding answer.

Good failure handling gives the LLM the context it needs to recover: it tells the model what went wrong, whether retrying makes sense, and what alternative approach it might try. This transforms failures from dead ends into learning signals that steer the agent toward a correct solution.

Anatomy of a Robust Tool Wrapper

The cleanest way to handle failures is to wrap every tool in a function that catches exceptions, normalizes the error into a structured message, and returns that message back to the LLM as if it were a tool result. The agent loop then continues, giving the model a chance to react.

Here's a minimal but complete agent loop with failure-aware tool execution:

import json
import time
import functools
from typing import Any, Callable

class ToolError(Exception):
    """Raised when a tool execution fails in a way the agent should know about."""
    def __init__(self, message: str, retryable: bool = False):
        super().__init__(message)
        self.message = message
        self.retryable = retryable

def with_error_handling(max_retries: int = 3, base_delay: float = 1.0):
    """Decorator that wraps a tool function with retry and error normalization."""
    def decorator(func: Callable) -> Callable:
        @functools.wraps(func)
        def wrapper(*args, **kwargs) -> Any:
            last_error = None
            for attempt in range(1, max_retries + 1):
                try:
                    return func(*args, **kwargs)
                except ToolError as e:
                    last_error = e
                    if not e.retryable or attempt == max_retries:
                        # Non-retryable, or we've exhausted retries.
                        # Return a structured error the LLM can read.
                        return {
                            "status": "error",
                            "error_type": "tool_error",
                            "message": e.message,
                            "retryable": e.retryable,
                            "attempts": attempt,
                        }
                    delay = base_delay * (2 ** (attempt - 1))
                    time.sleep(delay)
                except Exception as e:
                    # Unexpected error — don't retry, just report.
                    return {
                        "status": "error",
                        "error_type": "unexpected_error",
                        "message": f"{type(e).__name__}: {str(e)}",
                        "retryable": False,
                        "attempts": attempt,
                    }
            return {
                "status": "error",
                "error_type": "max_retries_exceeded",
                "message": str(last_error),
                "retryable": False,
                "attempts": max_retries,
            }
        return wrapper
    return decorator

The key design choice here is that errors are returned, not raised, once they reach the agent loop. The LLM consumes them as ordinary tool results. This keeps the loop simple and gives the model full agency to decide what to do next.

Defining Tools with Typed Failures

Now let's define a couple of real tools that use the wrapper. Notice how each tool distinguishes between retryable and non-retryable failures:

import requests

@with_error_handling(max_retries=3, base_delay=1.0)
def search_web(query: str, num_results: int = 5) -> dict:
    if not isinstance(query, str) or not query.strip():
        raise ToolError("query must be a non-empty string", retryable=False)
    if not isinstance(num_results, int) or num_results < 1 or num_results > 20:
        raise ToolError("num_results must be an integer between 1 and 20", retryable=False)

    try:
        resp = requests.get(
            "https://api.example-search.com/v1/search",
            params={"q": query, "count": num_results},
            headers={"Authorization": "Bearer MY_TOKEN"},
            timeout=10,
        )
    except requests.Timeout:
        raise ToolError("Search request timed out", retryable=True)
    except requests.ConnectionError:
        raise ToolError("Could not connect to search service", retryable=True)

    if resp.status_code == 429:
        raise ToolError("Rate limited by search API", retryable=True)
    if resp.status_code == 401:
        raise ToolError("Authentication failed — token may be expired", retryable=False)
    if resp.status_code >= 500:
        raise ToolError(f"Search service error (HTTP {resp.status_code})", retryable=True)

    data = resp.json()
    results = data.get("results", [])
    if not results:
        # Logical error: tool worked, but no results. Still useful to surface.
        return {"status": "success", "results": [], "message": "No results found."}
    return {"status": "success", "results": results}

@with_error_handling(max_retries=1, base_delay=0.5)
def query_database(sql: str) -> dict:
    if not sql.strip().lower().startswith("select"):
        raise ToolError("Only SELECT queries are permitted", retryable=False)
    # ... execute query ...
    return {"status": "success", "rows": []}

Feeding Errors Back to the LLM

The agent loop calls tools, formats their results (success or error) into messages, and appends them to the conversation. The LLM then decides whether to retry, try a different tool, or give up and answer the user.

def run_agent(user_prompt: str, llm_client, tools: dict, max_steps: int = 10):
    messages = [
        {"role": "system", "content": (
            "You are a helpful agent with access to tools. "
            "When a tool returns an error, read the 'message' field carefully. "
            "If 'retryable' is true, you may try again with corrected arguments. "
            "If 'retryable' is false, do not repeat the same call — try a different approach "
            "or explain the limitation to the user."
        )},
        {"role": "user", "content": user_prompt},
    ]

    for step in range(max_steps):
        response = llm_client.chat.completions.create(
            model="gpt-4o",
            messages=messages,
            tools=[tool_schema for tool_schema in TOOL_SCHEMAS],
        )
        choice = response.choices[0]
        messages.append(choice.message)

        if not choice.message.tool_calls:
            # No tool calls — agent is done, return final text.
            return choice.message.content

        for call in choice.message.tool_calls:
            tool_name = call.function.name
            args = json.loads(call.function.arguments)

            if tool_name not in tools:
                result = {
                    "status": "error",
                    "error_type": "unknown_tool",
                    "message": f"Tool '{tool_name}' does not exist.",
                    "retryable": False,
                }
            else:
                result = tools[tool_name](**args)

            messages.append({
                "role": "tool",
                "tool_call_id": call.id,
                "name": tool_name,
                "content": json.dumps(result),
            })

    return "Agent reached the maximum number of steps without finishing."

Notice the system prompt explicitly instructs the model on how to interpret error results. This is critical — without guidance, models often either ignore errors or get stuck in retry loops.

Best Practices

Always return errors as tool results, not exceptions. The agent loop should never crash because a tool failed. Wrap tool execution so failures become structured data the LLM can reason about.
Classify errors as retryable or not. Network blips deserve retries; schema violations do not. Communicate this distinction to the LLM in the error payload.
Use exponential backoff with jitter for retries. Retrying immediately against a rate-limited API just makes the limit worse. Add randomness to avoid thundering-herd effects.
Cap retries and steps. A misbehaving tool can trap the agent in an infinite loop. Hard limits on both retries per call and total agent steps prevent runaway token costs.
Validate arguments before calling the tool. Catch schema mismatches early and return a clear validation error rather than letting the tool crash deep in its internals.
Log every tool call and result. When an agent misbehaves, you need an audit trail of what was called, with what arguments, and what came back. This is invaluable for debugging.
Teach the model the error contract in the system prompt. Tell it what fields to read, when to retry, and when to pivot. Models follow these instructions surprisingly well when they're explicit.
Surface logical failures, not just exceptions. An empty search result or a zero-row query isn't an exception, but it's still a failure the agent needs to know about so it doesn't fabricate content.
Consider a fallback tool set. If a primary search API is down, the agent can be given a secondary one. Design tools so the model can choose alternatives when errors indicate a service is unavailable.

Conclusion

Handling tool execution failures is what separates a toy agent from a production-ready one. By wrapping every tool in error-aware logic, classifying failures as retryable or not, and feeding structured error messages back into the conversation, you give the LLM the information it needs to recover gracefully. The result is an agent that degrades smoothly under real-world conditions — retrying when it should, pivoting when it must, and always keeping the user informed rather than crashing or hallucinating. Build failure handling in from the start, and your agents will be dramatically more reliable when it matters.