← Back to DevBytes
Developer Tutorial July 18, 2026 9 min read

LangSmith for Agent Debugging: Complete Setup Guide

What is LangSmith?

LangSmith is an observability and testing platform from the creators of LangChain, designed specifically for LLM-powered applications and agents. It gives you a unified view across every step in your agent's reasoning pipeline, from initial prompt through tool selection and final response generation. Think of it as a specialized debugger that understands chains, prompts, tool calls, and retrieval steps.

At its core, LangSmith captures detailed traces – structured records of each operation your agent performs. A trace includes inputs, outputs, latency, token usage, metadata, and any errors that occurred. These traces are automatically organized by project, making it easy to filter, search, and analyze runs across development, staging, and production environments.

Why LangSmith Matters for Agent Debugging

Debugging a traditional function is straightforward: set a breakpoint, step through the code, inspect variables. Agents break that model. They involve non-deterministic LLM calls, dynamic tool selection, memory state, and multi-step reasoning loops. A single agent invocation can span 10–30 internal steps, each dependent on the previous output. When something goes wrong, you rarely get a clean stack trace – you get a vague "I'm sorry, I couldn't find that information" after a silent failure five steps earlier.

LangSmith fills this gap by giving you:

Getting Started: Prerequisites and Setup

To integrate LangSmith into your agent, you'll need an account and API key. The free tier includes a generous number of traces per month, so you can start without any upfront cost.

Step 1 – Create an account and get your API key:

Step 2 – Install the required Python package:

pip install -U langsmith langchain-core

If you're using the LangChain ecosystem heavily, you likely already have langchain-core. The langsmith package provides the client, tracing utilities, and decorators.

Step 3 – Configure environment variables:

export LANGCHAIN_TRACING_V2=true
export LANGCHAIN_API_KEY="your-api-key-here"
export LANGCHAIN_PROJECT="my-agent-project"

Setting LANGCHAIN_TRACING_V2=true activates the new tracing engine. The API key authenticates your application, and the project name organizes all traces under a single umbrella for easier filtering. For production, consider storing these in a secure secrets manager.

Instrumenting Your Agent with LangSmith Tracing

There are two primary ways to add tracing: automatic tracing for LangChain-based agents, and manual tracing using decorators or context managers. We'll cover both.

Automatic tracing with LangChain (zero code changes)

If you're building an agent with langgraph or the classic AgentExecutor, tracing often works out of the box once the environment variables are set. LangChain's built-in callback system automatically sends traces to LangSmith. 

import os
from langchain_openai import ChatOpenAI
from langgraph.prebuilt import create_react_agent

# Ensure env vars are set before importing / running
os.environ["LANGCHAIN_TRACING_V2"] = "true"
os.environ["LANGCHAIN_API_KEY"] = "your-api-key"
os.environ["LANGCHAIN_PROJECT"] = "my-agent-project"

llm = ChatOpenAI(model="gpt-4o", temperature=0)
tools = [...]  # your tool definitions
agent = create_react_agent(llm, tools)

# Each invoke automatically generates a trace
result = agent.invoke({"input": "Find the latest docs on tool calling"})
print(result["output"])

This is the fastest path. Run the script, and within seconds you'll see a complete trace appear in the LangSmith UI.

Manual tracing for custom agent loops

When you build an agent from scratch – a raw loop that manages its own state, calls LLMs, and executes tools – automatic tracing may not capture internal steps. Use the @traceable decorator or the traceable function context to gain full visibility.

Here's a minimal custom agent loop instrumented manually:

import json
from langsmith import traceable
from openai import OpenAI

client = OpenAI()

@traceable(
    run_type="tool",
    name="custom_search_tool",
    metadata={"source": "custom"}
)
def search_tool(query: str) -> str:
    # Simulated search – replace with real API
    return json.dumps({"results": ["doc1", "doc2"], "query": query})

@traceable(run_type="llm", name="reasoning_step")
def reason(state: dict) -> dict:
    messages = state.get("messages", [])
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=messages,
        temperature=0
    )
    content = response.choices[0].message.content
    return {"next_action": "tool" if "search" in content else "respond", "content": content}

def run_agent(user_input: str):
    state = {"messages": [{"role": "user", "content": user_input}]}
    for _ in range(10):  # safety limit
        step = reason(state)
        if step["next_action"] == "respond":
            return step["content"]
        # Call tool
        tool_result = search_tool(step["content"])
        state["messages"].append({"role": "user", "content": f"Tool output: {tool_result}"})
    return "Agent stopped due to step limit."

# Wrap the top-level entry point as well
traced_agent = traceable(run_agent, run_type="chain", name="custom_agent")

result = traced_agent("What are the latest papers on LLM agents?")
print(result)

Each decorated function automatically creates a child span within the overall trace. The trace tree will show custom_agent at the root, with reasoning_step spans and custom_search_tool spans nested inside. Inputs, outputs, and any exceptions are captured without additional logging code.

Advanced Configuration: Project Hierarchy and Tags

As your agent evolves, you'll want to organize traces beyond a flat project list. LangSmith supports tags and metadata key-value pairs on every run, enabling powerful filtering in the UI.

Add tags and metadata at invocation time:

result = traced_agent(
    "What is the capital of France?",
    langsmith_extra={
        "tags": ["production", "geo-query"],
        "metadata": {"deployment": "aws-lambda", "user_id": "u-456"}
    }
)

When using LangChain's invoke or ainvoke, you can pass these extras via the config parameter: 

from langchain_core.runnables import RunnableConfig

config = RunnableConfig(
    tags=["eval-set-1"],
    metadata={"experiment": "tool-optimization"}
)
agent.invoke({"input": "..."}, config=config)

In the LangSmith UI, you can then filter runs by tag (e.g., tag = "production") or by metadata fields. This is invaluable when comparing different versions or isolating a specific user's session that triggered a bug.

Debugging Agent Traces in the UI

Once traces flow into LangSmith, the web interface becomes your primary debugging surface. Here's how to navigate and extract actionable information.

Navigating a trace

Click any run in your project's run list. The trace view displays a hierarchical tree on the left and a detail panel on the right. The root span represents the entire agent invocation; children are LLM calls, tool executions, retriever steps, and custom functions.

Key elements to inspect:

Common debugging scenarios

Scenario 1: Agent enters an infinite loop. Look at the trace tree. You'll see the same tool being called repeatedly with identical inputs. The issue is usually in the reasoning step – the LLM doesn't recognize it has already tried that tool. Fix the prompt to include a history of previous actions.

Scenario 2: Tool returns an error but agent continues. Find the tool span marked as error. Check the output: maybe the tool returned a string the agent couldn't parse. You can then improve error handling in your tool wrapper or the agent's parsing logic.

Scenario 3: Final answer is wrong, but all steps succeeded. Trace backwards from the final output. Look at the last reasoning step's input – did it include the correct tool outputs? Often, the context window truncation or incorrect message ordering causes information loss. The trace shows the exact messages array.

Using the LangSmith Client for Programmatic Debugging

The web UI is great for manual inspection, but for automated testing or bulk analysis you'll want the Python client. LangSmith provides a full SDK to query runs, fetch traces, and export data.

Install the client (already included in langsmith):

from langsmith import Client

client = Client()

Fetching recent runs

runs = list(client.list_runs(
    project_name="my-agent-project",
    run_type="chain",
    start_time="2025-03-01T00:00:00Z",  # optional filter
    error=True  # only failed runs
))
for run in runs:
    print(run.name, run.error)

Drilling into a specific trace

trace = client.read_run("run-id-from-ui-or-list")
print("Input:", trace.inputs)
print("Output:", trace.outputs)
print("Error:", trace.error)
# Child spans
children = list(client.list_runs(parent_run_id=trace.id))
for child in children:
    print(f"  {child.name} - {child.run_type}")

Exporting data for offline analysis

import pandas as pd

runs_data = []
for run in client.list_runs(project_name="my-agent-project"):
    runs_data.append({
        "id": run.id,
        "name": run.name,
        "latency_ms": (run.end_time - run.start_time).total_seconds() * 1000,
        "token_usage": run.total_tokens,
        "error": bool(run.error)
    })
df = pd.DataFrame(runs_data)
print(df.describe())

This programmatic access allows you to build regression test suites, cost monitoring dashboards, or automated alerts when error rates spike above a threshold.

Best Practices for Agent Debugging with LangSmith

Over time, teams develop patterns that make LangSmith truly effective. Adopt these habits from day one.

Conclusion

LangSmith transforms agent debugging from a frustrating grep-and-guess exercise into a structured, visual process. By capturing every step, input, output, and error in a searchable trace tree, it lets you isolate failures in seconds rather than hours. The setup is lightweight – often just a few environment variables – and scales from a single developer's laptop to a full production cluster. Start by instrumenting your agent's core reasoning loop, adopt consistent tagging, and use both the UI and the client SDK to build a debugging workflow that grows with your system's complexity. Once you've experienced the clarity of a fully traced agent run, you'll wonder how you ever debugged without it.

— Ad —

Google AdSense will appear here after approval

← Back to all articles