How to Evaluate Tool Selection Accuracy in Agents

How to Evaluate Tool Selection Accuracy in Agents

Tool selection accuracy is one of the most critical metrics when building LLM-based agents that can call external tools. An agent might reason beautifully, but if it picks the wrong tool for the job — or fails to pick one at all — the entire pipeline breaks. This tutorial walks through what tool selection accuracy is, why it matters, how to measure it, and how to build a practical evaluation harness you can reuse across projects.

What Is Tool Selection Accuracy?

Tool selection accuracy measures how often an agent chooses the correct tool (or set of tools) when presented with a user query. It compares the tools the agent actually invoked against a ground-truth expectation defined by a labeled dataset. In its simplest form, it is a binary classification metric per example: did the agent pick the right tool, yes or no?

More nuanced evaluations also consider argument correctness, ordering, and whether the agent avoided calling irrelevant tools. But at its core, the metric answers a focused question: given this input, did the agent route to the right capability?

Why It Matters

• User trust: Calling the wrong tool produces confusing, broken responses that erode confidence faster than a bad text completion.
• Cost and latency: Unnecessary tool calls burn tokens, hit rate limits, and add round-trips that slow down responses.
• Safety: Some tools have side effects (sending emails, deleting records, executing SQL). Misrouting to a destructive tool is a real risk.
• Debuggability: A measurable tool-selection metric gives you a concrete signal to regress-test against when you change prompts, models, or tool descriptions.
• Composition: In multi-agent systems, downstream agents depend on upstream routing. A wrong selection cascades into wrong outputs.

How to Use It: Building an Evaluation Harness

The standard approach is to assemble a golden dataset of (query, expected_tools) pairs, run the agent against each query, capture the tools it actually called, and compute accuracy plus supporting metrics. Let's build this step by step.

Step 1: Define the Golden Dataset

Your dataset should cover positive cases (the tool should be called), negative cases (the tool should NOT be called), and multi-tool cases. Diversity matters more than volume — a few hundred well-chosen examples usually beats thousands of redundant ones.

golden_dataset = [
    {
        "id": "q1",
        "query": "What's the weather in Tokyo right now?",
        "expected_tools": ["get_weather"],
        "expected_args": {"city": "Tokyo"}
    },
    {
        "id": "q2",
        "query": "Send an email to the engineering team about the deploy.",
        "expected_tools": ["send_email"],
        "expected_args": {"recipient": "engineering"}
    },
    {
        "id": "q3",
        "query": "Search the web for recent papers on retrieval-augmented generation, then summarize them.",
        "expected_tools": ["web_search", "summarize"],
        "expected_args": {}
    },
    {
        "id": "q4",
        "query": "Thanks, that's all I needed.",
        "expected_tools": [],
        "expected_args": {}
    }
]

Step 2: Capture the Agent's Tool Calls

You need an instrumented agent runner that records every tool invocation. The cleanest pattern is to wrap the agent's tool-calling loop so each call is logged to a trace.

import json
from dataclasses import dataclass, field

@dataclass
class AgentTrace:
    query: str
    tool_calls: list = field(default_factory=list)

    def add_call(self, name: str, args: dict):
        self.tool_calls.append({"name": name, "args": args})


def run_agent_with_tracing(agent, query: str) -> AgentTrace:
    trace = AgentTrace(query=query)

    def tool_wrapper(tool_name, tool_fn):
        def wrapped(**kwargs):
            trace.add_call(tool_name, kwargs)
            return tool_fn(**kwargs)
        return wrapped

    # Replace each registered tool with its traced version
    original_tools = agent.tools
    agent.tools = {
        name: tool_wrapper(name, fn)
        for name, fn in original_tools.items()
    }

    try:
        agent.run(query)
    finally:
        agent.tools = original_tools  # restore

    return trace

Step 3: Compute Accuracy Metrics

Now compare the trace against the golden expectation. We'll compute three complementary metrics: exact-match accuracy, precision, and recall over tool names.

def evaluate_tool_selection(dataset, agent):
    results = []
    for example in dataset:
        trace = run_agent_with_tracing(agent, example["query"])
        actual = [c["name"] for c in trace.tool_calls]
        expected = example["expected_tools"]

        actual_set = set(actual)
        expected_set = set(expected)

        tp = len(actual_set & expected_set)
        fp = len(actual_set - expected_set)
        fn = len(expected_set - actual_set)

        precision = tp / (tp + fp) if (tp + fp) else 1.0
        recall = tp / (tp + fn) if (tp + fn) else 1.0
        exact_match = (actual_set == expected_set)

        results.append({
            "id": example["id"],
            "query": example["query"],
            "actual": actual,
            "expected": expected,
            "precision": precision,
            "recall": recall,
            "exact_match": exact_match
        })
    return results


def summarize(results):
    n = len(results)
    em = sum(r["exact_match"] for r in results) / n
    avg_p = sum(r["precision"] for r in results) / n
    avg_r = sum(r["recall"] for r in results) / n
    f1 = (2 * avg_p * avg_r / (avg_p + avg_r)) if (avg_p + avg_r) else 0.0
    return {
        "exact_match_accuracy": em,
        "avg_precision": avg_p,
        "avg_recall": avg_r,
        "avg_f1": f1,
        "n_examples": n
    }

Step 4: Inspect Failures

Aggregate numbers are useful for tracking regressions, but the real value comes from inspecting failures. Filter the results to find cases where the agent picked the wrong tool, then categorize them.

def failure_report(results):
    failures = [r for r in results if not r["exact_match"]]
    report = []
    for f in failures:
        false_positives = set(f["actual"]) - set(f["expected"])
        false_negatives = set(f["expected"]) - set(f["actual"])
        report.append({
            "id": f["id"],
            "query": f["query"],
            "false_positives": list(false_positives),
            "false_negatives": list(false_negatives)
        })
    return report

Common failure patterns you'll see: tools with overlapping descriptions confuse the agent, vague queries trigger no tool when one is needed, and chatty follow-up messages incorrectly trigger tools. Each pattern points to a specific fix in your tool descriptions or system prompt.

Step 5: Evaluate Argument Correctness (Optional but Powerful)

Selecting the right tool is only half the battle. The agent also needs to pass the right arguments. You can extend the harness to validate arguments against expected values or schemas.

def evaluate_args(results, dataset):
    expected_by_id = {e["id"]: e["expected_args"] for e in dataset}
    arg_results = []
    for r in results:
        exp_args = expected_by_id.get(r["id"], {})
        if not exp_args:
            continue
        # Match each expected arg against the corresponding call
        for call in r["actual"]:
            # In a real harness you'd track args per call, not just names
            pass
        # Simplified: check if expected key/value pairs appear anywhere
        arg_results.append({
            "id": r["id"],
            "expected_args": exp_args
        })
    return arg_results

For production systems, consider using an LLM-as-judge to score argument quality on a 1–5 scale, since exact string matching is often too strict for free-form inputs like email bodies or search queries.

Best Practices

• Keep a frozen golden set. Version it in source control so you can detect regressions when prompts or models change.
• Include negative examples. Queries that should trigger no tool are just as important as ones that should. Agents that over-call tools are a real problem.
• Test tool description changes in isolation. A single word in a tool description can swing accuracy by double digits. Run the eval before and after.
• Track per-tool recall. Aggregate accuracy hides weak tools. Break down recall by tool name to find which capabilities the agent struggles to route to.
• Use stratified sampling. Balance easy, hard, ambiguous, and adversarial queries so the metric reflects real-world distribution.
• Log traces, not just scores. Store the full agent trace for each evaluation run so you can debug failures without re-running the agent.
• Run evals in CI. Wire the harness into your test pipeline and fail builds when exact-match accuracy drops below a threshold.
• Watch for model drift. Re-run the full eval whenever the underlying model is updated, even if your code hasn't changed.
• Combine with end-to-end evaluation. Tool selection accuracy is a leading indicator, but pair it with final-answer quality metrics for the full picture.

Conclusion

Evaluating tool selection accuracy gives you a concrete, measurable signal for one of the most failure-prone parts of agent systems. By building a golden dataset, instrumenting your agent to capture tool calls, and computing exact-match accuracy alongside precision and recall, you create a repeatable harness that catches regressions before users do. The metrics themselves are simple — the discipline of maintaining the dataset and acting on failure reports is what makes the difference. Start small with a few dozen examples, wire the harness into CI, and grow the golden set as your agent's tool inventory expands. Over time, this becomes one of the most reliable levers you have for shipping trustworthy agent behavior.