← Back to DevBytes
AI August 12, 2026 10 min read

Testing LangChain Applications: Unit Tests to Integration

Testing LangChain Applications: From Unit Tests to Integration

LangChain makes it remarkably easy to stitch together large language models, prompts, tools, and memory into powerful applications. But that same flexibility introduces a tricky testing problem: your app depends on nondeterministic models, external APIs, vector stores, and chained logic that can fail in surprising ways. Without a deliberate testing strategy, you end up debugging production traffic instead of shipping features.

This tutorial walks through a pragmatic approach to testing LangChain applications — starting with pure unit tests for individual components, moving through prompt and output validation, and finishing with integration tests that exercise the full chain. You'll come away with patterns you can drop directly into your own codebase.

Why Testing LangChain Apps Is Different

Traditional applications are deterministic: given the same input, you get the same output. LLM applications break that assumption. A chain that produces a perfect answer in testing might return a slightly different phrasing — or a hallucination — in production. On top of that, LangChain code typically orchestrates several moving parts:

A good testing strategy isolates each of these layers so you can catch regressions quickly and cheaply, then validates the assembled system with a smaller set of integration tests.

Structuring Your Test Pyramid

For LangChain apps, adapt the classic test pyramid into three tiers:

The vast majority of your tests should live in the first two tiers. Integration tests are valuable but slow and costly, so keep them focused on critical user-facing flows.

Setting Up the Project

We'll use pytest with pytest-asyncio since many LangChain components are async. Install the dependencies:

pip install pytest pytest-asyncio pytest-mock langchain langchain-openai

Add a pytest.ini at your project root:

[pytest]
asyncio_mode = auto
testpaths = tests
python_files = test_*.py

A typical project layout looks like this:

my_app/
├── app/
│   ├── chains.py
│   ├── prompts.py
│   ├── tools.py
│   └── parsers.py
├── tests/
│   ├── unit/
│   ├── component/
│   └── integration/
└── pytest.ini

Unit Testing Prompts and Parsers

Prompts and output parsers are pure logic — perfect candidates for fast unit tests. Let's say you have a prompt template that builds a summarization request:

# app/prompts.py
from langchain_core.prompts import ChatPromptTemplate

SUMMARIZE_PROMPT = ChatPromptTemplate.from_messages([
    ("system", "You are a concise summarizer. Respond in {word_count} words or fewer."),
    ("user", "Summarize the following text:\n\n{text}"),
])

def build_summary_messages(text: str, word_count: int = 50):
    return SUMMARIZE_PROMPT.format_messages(text=text, word_count=word_count)

Test that the prompt assembles correctly without ever calling a model:

# tests/unit/test_prompts.py
from app.prompts import build_summary_messages

def test_build_summary_messages_includes_text():
    messages = build_summary_messages("LangChain is a framework.", word_count=30)

    assert len(messages) == 2
    assert messages[0].role == "system"
    assert "30" in messages[0].content
    assert "LangChain is a framework." in messages[1].content

def test_build_summary_messages_rejects_missing_text():
    import pytest
    with pytest.raises(Exception):
        build_summary_messages("")

Output parsers are equally testable. Suppose you parse a model's response into structured JSON:

# app/parsers.py
import json
from langchain_core.exceptions import OutputParserException

class ActionParser:
    def parse(self, text: str) -> dict:
        try:
            data = json.loads(text)
        except json.JSONDecodeError as e:
            raise OutputParserException(f"Invalid JSON: {e}") from e
        if "action" not in data:
            raise OutputParserException("Missing 'action' key")
        return data
# tests/unit/test_parsers.py
import pytest
from langchain_core.exceptions import OutputParserException
from app.parsers import ActionParser

def test_parse_valid_json():
    parser = ActionParser()
    result = parser.parse('{"action": "search", "query": "cats"}')
    assert result["action"] == "search"

def test_parse_invalid_json_raises():
    parser = ActionParser()
    with pytest.raises(OutputParserException):
        parser.parse("not json at all")

def test_parse_missing_action_raises():
    parser = ActionParser()
    with pytest.raises(OutputParserException):
        parser.parse('{"foo": "bar"}')

These tests run in milliseconds and catch the most common regressions: malformed prompts, broken variable substitution, and parser edge cases.

Unit Testing Custom Tools

Custom tools in LangChain are just functions with a schema. Test them like any other function — mock external dependencies, assert on inputs and outputs.

# app/tools.py
from langchain_core.tools import tool
import requests

@tool
def get_weather(city: str) -> str:
    """Return the current weather for a city."""
    resp = requests.get(f"https://api.weather.example.com/{city}", timeout=5)
    resp.raise_for_status()
    data = resp.json()
    return f"{data['temp']}C, {data['condition']}"
# tests/unit/test_tools.py
from unittest.mock import patch, MagicMock
from app.tools import get_weather

@patch("app.tools.requests.get")
def test_get_weather_returns_formatted_string(mock_get):
    mock_get.return_value = MagicMock(
        json=lambda: {"temp": 22, "condition": "sunny"},
        status_code=200,
    )
    mock_get.return_value.raise_for_status = MagicMock()

    result = get_weather.invoke({"city": "Berlin"})
    assert result == "22C, sunny"
    mock_get.assert_called_once_with("https://api.weather.example.com/Berlin", timeout=5)

Notice we use .invoke() rather than calling the function directly — this exercises the tool through LangChain's Runnable interface, which is what your chain will actually use.

Component Testing: Mocking the LLM

Once you've tested the pieces, the next layer validates how they fit together — without paying for real LLM calls. LangChain provides FakeListLLM and FakeMessagesListLLM for exactly this purpose.

# app/chains.py
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

QA_PROMPT = ChatPromptTemplate.from_messages([
    ("system", "Answer the question based only on the context."),
    ("user", "Context: {context}\n\nQuestion: {question}"),
])

def build_qa_chain(llm):
    return QA_PROMPT | llm | StrOutputParser()
# tests/component/test_qa_chain.py
from langchain_core.language_models.fake_chat_models import FakeMessagesListChatModel
from langchain_core.messages import AIMessage
from app.chains import build_qa_chain

def test_qa_chain_passes_context_and_question():
    fake_llm = FakeMessagesListChatModel(responses=[
        AIMessage(content="Paris is the capital of France.")
    ])
    chain = build_qa_chain(fake_llm)

    result = chain.invoke({
        "context": "France is a country in Europe. Its capital is Paris.",
        "question": "What is the capital of France?"
    })

    assert result == "Paris is the capital of France."
    # Verify the prompt was assembled correctly
    prompt_value = fake_llm.calls[0].messages
    assert "capital of France" in prompt_value[1].content

This test confirms the chain wires the prompt, model, and parser together correctly — all without a network call. If someone refactors the chain and accidentally drops the context variable, this test fails immediately.

Testing Agent Tool Selection

Agents are trickier because the model decides which tool to call. With a fake LLM, you can script the model's responses and assert that the agent invokes the right tools in the right order.

# tests/component/test_agent.py
from langchain_core.language_models.fake_chat_models import FakeMessagesListChatModel
from langchain_core.messages import AIMessage
from langchain.agents import create_tool_calling_agent, AgentExecutor
from langchain_core.prompts import ChatPromptTemplate
from unittest.mock import patch, MagicMock

def test_agent_calls_weather_tool():
    fake_llm = FakeMessagesListChatModel(responses=[
        AIMessage(
            content="",
            tool_calls=[{
                "name": "get_weather",
                "args": {"city": "Berlin"},
                "id": "call_1",
            }]
        ),
        AIMessage(content="The weather in Berlin is 22C, sunny."),
    ])

    prompt = ChatPromptTemplate.from_messages([
        ("system", "You are a helpful assistant."),
        ("user", "{input}"),
        ("placeholder", "{agent_scratchpad}"),
    ])

    with patch("app.tools.requests.get") as mock_get:
        mock_get.return_value = MagicMock(
            json=lambda: {"temp": 22, "condition": "sunny"},
            status_code=200,
        )
        mock_get.return_value.raise_for_status = MagicMock()

        from app.tools import get_weather
        agent = create_tool_calling_agent(fake_llm, [get_weather], prompt)
        executor = AgentExecutor(agent=agent, tools=[get_weather], verbose=False)

        result = executor.invoke({"input": "What's the weather in Berlin?"})

    assert "22C" in result["output"]
    mock_get.assert_called_once()

By scripting the model's tool calls, you can deterministically test the agent's orchestration — including error paths where a tool raises an exception.

Integration Testing the Full Chain

Integration tests exercise the real system. They're slower and cost tokens, so be deliberate about what you test. The key challenge is nondeterminism: you can't assert exact strings. Instead, assert on structural properties.

# tests/integration/test_qa_integration.py
import os
import pytest
from langchain_openai import ChatOpenAI
from app.chains import build_qa_chain

@pytest.fixture(scope="module")
def llm():
    return ChatOpenAI(model="gpt-4o-mini", temperature=0)

@pytest.fixture(scope="module")
def chain(llm):
    return build_qa_chain(llm)

def test_qa_returns_correct_answer(chain):
    result = chain.invoke({
        "context": "The Eiffel Tower is located in Paris, France.",
        "question": "Where is the Eiffel Tower?"
    })
    assert "Paris" in result
    assert len(result) < 500  # sanity check on verbosity

def test_qa_handles_empty_context(chain):
    result = chain.invoke({
        "context": "",
        "question": "Where is the Eiffel Tower?"
    })
    assert isinstance(result, str)
    assert len(result) > 0

Setting temperature=0 reduces (but does not eliminate) variance. For more robust assertions, consider these strategies:

Using an LLM-as-Judge for Integration Assertions

# tests/integration/judge.py
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate

JUDGE_PROMPT = ChatPromptTemplate.from_messages([
    ("system", "You are grading answers. Respond with only 'PASS' or 'FAIL'."),
    ("user", "Question: {question}\nExpected: {expected}\nActual: {actual}\nDoes the actual answer correctly address the question?"),
])

def grade(question: str, expected: str, actual: str) -> bool:
    llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
    chain = JUDGE_PROMPT | llm
    result = chain.invoke({"question": question, "expected": expected, "actual": actual})
    return "PASS" in result.content.strip().upper()
# tests/integration/test_qa_with_judge.py
import pytest
from langchain_openai import ChatOpenAI
from app.chains import build_qa_chain
from tests.integration.judge import grade

@pytest.fixture(scope="module")
def chain():
    return build_qa_chain(ChatOpenAI(model="gpt-4o-mini", temperature=0))

def test_qa_answer_quality(chain):
    result = chain.invoke({
        "context": "Photosynthesis converts light energy into chemical energy stored in glucose.",
        "question": "What does photosynthesis do?"
    })
    assert grade(
        question="What does photosynthesis do?",
        expected="Converts light energy into chemical energy.",
        actual=result,
    )

This pattern lets you assert semantic correctness even when the exact wording varies run to run.

Testing Retrievers and Vector Stores

Retrieval-augmented generation (RAG) chains depend on a vector store returning relevant documents. For unit tests, mock the retriever entirely:

# tests/component/test_rag_chain.py
from langchain_core.language_models.fake_chat_models import FakeMessagesListChatModel
from langchain_core.messages import AIMessage, Document
from unittest.mock import MagicMock
from app.chains import build_rag_chain

def test_rag_chain_uses_retrieved_documents():
    fake_retriever = MagicMock()
    fake_retriever.invoke.return_value = [
        Document(page_content="The company was founded in 2015."),
    ]
    fake_llm = FakeMessagesListChatModel(responses=[
        AIMessage(content="The company was founded in 2015.")
    ])

    chain = build_rag_chain(fake_retriever, fake_llm)
    result = chain.invoke({"question": "When was the company founded?"})

    assert "2015" in result
    fake_retriever.invoke.assert_called_once()

For integration tests against a real vector store, use a small fixture corpus and assert that the retriever returns the expected documents for known queries. Consider running a local instance of Chroma or Qdrant in a Docker container to keep tests hermetic.

Recording and Replaying LLM Calls

For component tests that need realistic model behavior, consider recording real LLM responses once and replaying them. The langchain-core chat_models can be wrapped with a caching layer, or you can use tools like vcrpy to record HTTP traffic. This gives you realistic outputs in CI without paying for tokens on every run.

# tests/conftest.py
import json
import os
from langchain_openai import ChatOpenAI

RECORDINGS_DIR = "tests/recordings"

class ReplayChatModel:
    """A simple replay model that returns canned responses from a JSON file."""
    def __init__(self, recording_file: str):
        with open(os.path.join(RECORDINGS_DIR, recording_file)) as f:
            self.responses = json.load(f)
        self.index = 0

    def invoke(self, messages, **kwargs):
        response = self.responses[self.index]
        self.index += 1
        from langchain_core.messages import AIMessage
        return AIMessage(content=response)

@pytest.fixture
def replay_llm():
    return ReplayChatModel("qa_chain_responses.json")

This is a simplified example — production setups typically use VCR-style cassettes that match on request payloads. The principle is the same: capture once, replay forever, refresh when prompts change.

Best Practices

Organizing Test Runs with Markers

# pytest.ini
[pytest]
asyncio_mode = auto
testpaths = tests
markers =
    unit: fast, isolated tests
    component: chain-level tests with mocked LLM
    integration: end-to-end tests against real services
# Run only fast tests in CI pull-request checks
pytest -m "not integration"

# Run the full suite on merge to main
pytest

Conclusion

Testing LangChain applications requires a shift in mindset: you can't assert exact outputs the way you would in a traditional codebase, but you can build a robust safety net by testing each layer in isolation and validating the assembled system with tolerant, semantically aware assertions. Start with fast unit tests for prompts, parsers, and tools; add component tests that mock the LLM to verify orchestration; and finish with a focused set of integration tests that exercise real models against critical user flows. By treating prompts as code, injecting dependencies for testability, and using techniques like LLM-as-judge and recorded replays, you can ship LangChain features with confidence — even when the underlying models are inherently unpredictable.

— Ad —

Google AdSense will appear here after approval

← Back to all articles