Agent Memory Architectures with MCP (Model Context Protocol): Complete Guide

Agent Memory Architectures with MCP (Model Context Protocol): Complete Guide

Building agents that remember across sessions, reason over long horizons, and share context with tools is one of the hardest problems in applied LLM engineering. The Model Context Protocol (MCP) — an open standard introduced by Anthropic — gives us a unified way to expose context, tools, and prompts to LLMs. When combined with a thoughtful memory architecture, MCP turns a stateless chatbot into a persistent, capable agent. This guide walks through everything you need to know to design, build, and ship memory-backed agents using MCP.

What Is the Model Context Protocol?

MCP is a client-server protocol that standardizes how applications provide context to LLMs. An MCP server exposes three primitives: resources (read-only data), tools (executable functions), and prompts (reusable templates). An MCP client — typically an agent runtime or an IDE like Claude Desktop — connects to one or more servers and composes their offerings into the model's context window.

The key insight is that MCP decouples context sources from the model. Instead of hard-coding retrieval logic or tool definitions into your agent, you wire them up as MCP servers. This makes memory, search, databases, and APIs pluggable.

Why Memory Architectures Matter

LLMs are stateless. Every request is independent, and the context window is finite. Without an explicit memory architecture, your agent will:

• Forget user preferences between sessions
• Lose track of long-running tasks and intermediate results
• Repeat work it already completed
• Fail to ground answers in historical context
• Scale poorly as conversation history grows beyond the context window

A well-designed memory architecture solves these problems by externalizing state. MCP then becomes the transport layer that feeds the right memories into the model at the right time.

Types of Agent Memory

Before writing code, you need a mental model. Most production agents use three tiers of memory, inspired by cognitive science but adapted for LLMs.

Short-Term (Working) Memory

This is the current conversation — the messages exchanged in the active session. It lives in the context window and is discarded when the session ends. Short-term memory is essential for coherence within a single task but expensive to maintain as it grows.

Long-Term (Episodic) Memory

Episodic memory stores past interactions, outcomes, and events. When a user returns days later, episodic memory lets the agent recall what happened previously. This is typically backed by a vector database or a time-series store.

Semantic (Knowledge) Memory

Semantic memory holds distilled facts: user preferences, entity attributes, policies, and learned rules. Unlike episodic memory, which stores raw events, semantic memory stores compact, queryable knowledge. This is where structured stores like graphs or key-value databases shine.

Procedural Memory

Procedural memory captures how to do things — workflows, tool sequences, and learned skills. In MCP terms, this often maps to saved prompts and tool compositions that the agent can reuse.

Designing a Memory Architecture with MCP

The cleanest pattern is to expose each memory tier as its own MCP server. The agent client connects to all of them and decides what to pull into context per turn. This separation gives you independent scaling, storage backends, and access control.

A typical topology looks like this:

• Working Memory Server — manages the active session buffer, summarization, and truncation.
• Episodic Memory Server — stores and retrieves past conversations via vector search.
• Semantic Memory Server — stores user facts, entities, and preferences in a structured store.
• Procedural Memory Server — exposes reusable prompts and learned workflows.

Setting Up an MCP Server

MCP servers can be written in TypeScript or Python. We will use the official Python SDK. Install it first:

pip install mcp anthropic

Here is a minimal MCP server that exposes a semantic memory store backed by a simple in-memory dictionary. In production you would swap this for Redis, Postgres, or a graph database.

from mcp.server.fastmcp import FastMCP

mcp = FastMCP("semantic-memory")

# In-memory store; replace with a real database in production
_store: dict[str, str] = {}

@mcp.tool()
def remember(key: str, value: str) -> str:
    """Store a fact about the user or world."""
    _store[key] = value
    return f"Remembered: {key} = {value}"

@mcp.tool()
def recall(key: str) -> str:
    """Retrieve a stored fact by key."""
    return _store.get(key, f"No memory found for key: {key}")

@mcp.tool()
def list_facts() -> str:
    """List all stored facts."""
    if not _store:
        return "No facts stored yet."
    return "\n".join(f"- {k}: {v}" for k, v in _store.items())

if __name__ == "__main__":
    mcp.run()

Run the server with python semantic_memory_server.py. It will start listening on stdio, which is the default transport for local MCP servers.

Building an Episodic Memory Server

Episodic memory needs semantic search so the agent can find relevant past conversations by meaning, not just keyword. We will use a lightweight in-memory vector store with sentence embeddings. For production, swap in Qdrant, Pinecone, pgvector, or Chroma.

import json
import numpy as np
from mcp.server.fastmcp import FastMCP

mcp = FastMCP("episodic-memory")

# Simple embedding: hash-based bag of words (replace with real embeddings)
def embed(text: str) -> np.ndarray:
    vec = np.zeros(512)
    for word in text.lower().split():
        vec[hash(word) % 512] += 1.0
    norm = np.linalg.norm(vec)
    return vec / norm if norm > 0 else vec

_episodes: list[dict] = []

@mcp.tool()
def log_episode(summary: str, details: str) -> str:
    """Store a past interaction with a summary and full details."""
    ep_id = len(_episodes) + 1
    _episodes.append({
        "id": ep_id,
        "summary": summary,
        "details": details,
        "embedding": embed(summary + " " + details)
    })
    return f"Logged episode #{ep_id}"

@mcp.tool()
def search_episodes(query: str, top_k: int = 3) -> str:
    """Find past episodes relevant to the query."""
    if not _episodes:
        return "No episodes stored yet."
    q_vec = embed(query)
    scored = []
    for ep in _episodes:
        score = float(np.dot(q_vec, ep["embedding"]))
        scored.append((score, ep))
    scored.sort(key=lambda x: x[0], reverse=True)
    results = scored[:top_k]
    return json.dumps([
        {"id": ep["id"], "summary": ep["summary"], "score": round(s, 4)}
        for s, ep in results
    ], indent=2)

if __name__ == "__main__":
    mcp.run()

This server gives the agent two tools: one to record what happened, and one to retrieve relevant history. The agent decides when to call them based on the user's request.

Working Memory and Summarization

Working memory is the trickiest tier because it must fit in the context window. The standard approach is a sliding window with periodic summarization. When the conversation exceeds a threshold, older messages are compressed into a summary and stored in episodic memory.

from mcp.server.fastmcp import FastMCP

mcp = FastMCP("working-memory")

MAX_MESSAGES = 20
_messages: list[dict] = []
_summary: str = ""

@mcp.tool()
def add_message(role: str, content: str) -> str:
    """Append a message to the active session."""
    _messages.append({"role": role, "content": content})
    if len(_messages) > MAX_MESSAGES:
        return "BUFFER_FULL"
    return "OK"

@mcp.tool()
def get_context() -> str:
    """Return the current summary plus recent messages."""
    parts = []
    if _summary:
        parts.append(f"[Summary of earlier conversation]\n{_summary}")
    for msg in _messages:
        parts.append(f"{msg['role']}: {msg['content']}")
    return "\n\n".join(parts)

@mcp.tool()
def compact(summary: str) -> str:
    """Replace the message buffer with a summary, clearing old messages."""
    global _summary, _messages
    _summary = summary
    _messages = []
    return "Compacted. Buffer reset."

if __name__ == "__main__":
    mcp.run()

The agent client checks the return value of add_message. When it sees BUFFER_FULL, it asks the model to summarize the conversation, calls compact with that summary, and continues. The old messages can also be logged to episodic memory before compaction.

Connecting the Agent to Multiple MCP Servers

Now we build the agent client that orchestrates all three memory servers. The Python SDK provides a ClientSession for connecting to servers over stdio.

import asyncio
from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client

SERVERS = {
    "semantic": "semantic_memory_server.py",
    "episodic": "episodic_memory_server.py",
    "working":  "working_memory_server.py",
}

async def connect_all():
    sessions = {}
    for name, script in SERVERS.items():
        params = StdioServerParameters(
            command="python",
            args=[script],
        )
        read, write = await stdio_client(params).__aenter__()
        session = ClientSession(read, write)
        await session.initialize()
        sessions[name] = session
    return sessions

async def main():
    sessions = await connect_all()

    # Store a user preference
    result = await sessions["semantic"].call_tool(
        "remember", {"key": "preferred_language", "value": "Python"}
    )
    print(result)

    # Log an episode
    await sessions["episodic"].call_tool(
        "log_episode",
        {"summary": "User asked about MCP memory",
         "details": "Explained short, long, and semantic memory tiers."}
    )

    # Search history
    hits = await sessions["episodic"].call_tool(
        "search_episodes", {"query": "MCP memory", "top_k": 2}
    )
    print(hits)

asyncio.run(main())

In a full agent loop, you would interleave these tool calls with LLM completions. The model receives the available tools from all connected MCP servers, decides which to call, and you route the calls back through the appropriate session.

Integrating with an LLM Loop

Here is a simplified agent loop that uses the Anthropic API and routes tool calls to the correct MCP server.

import anthropic

client = anthropic.Anthropic()

async def agent_turn(user_input: str, sessions: dict):
    # Gather tools from all servers
    all_tools = []
    for name, session in sessions.items():
        resp = await session.list_tools()
        for tool in resp.tools:
            all_tools.append({
                "name": f"{name}__{tool.name}",
                "description": tool.description,
                "input_schema": tool.inputSchema,
            })

    messages = [{"role": "user", "content": user_input}]

    while True:
        response = client.messages.create(
            model="claude-sonnet-4-5",
            max_tokens=4096,
            tools=[{"type": "custom", "custom_tool": t} for t in all_tools],
            messages=messages,
        )
        messages.append({"role": "assistant", "content": response.content})

        if response.stop_reason != "tool_use":
            return response.content

        for block in response.content:
            if block.type != "tool_use":
                continue
            server_name, tool_name = block.name.split("__", 1)
            result = await sessions[server_name].call_tool(
                tool_name, block.input
            )
            messages.append({
                "role": "user",
                "content": [{
                    "type": "tool_result",
                    "tool_use_id": block.id,
                    "content": str(result),
                }],
            })

This loop continues until the model stops requesting tools and returns a final answer. Each memory tier is accessible through the same uniform interface.

Best Practices

Separate Memory Tiers into Separate Servers

Resist the urge to put all memory in one server. Separate servers let you scale storage independently, apply different retention policies, and debug each tier in isolation. They also make it easy to swap backends — for example, moving episodic memory from an in-memory list to Qdrant without touching semantic memory.

Summarize Aggressively, Store Selectively

Not every message deserves a place in long-term memory. Use the model to decide what is worth remembering. A good pattern is a dedicated reflect tool that takes the current conversation, extracts salient facts and episodes, and writes them to the appropriate servers. This keeps your stores clean and retrieval fast.

Use Structured Schemas for Semantic Memory

Free-text facts are hard to query. Prefer structured keys and typed values. For example, store {"user.timezone": "UTC-5", "user.role": "engineer"} rather than a paragraph of prose. If you need richer relationships, use a graph store and expose Cypher or SPARQL queries as MCP tools.

Version Your Memory Schemas

As your agent evolves, the shape of stored memories will change. Include a schema version in every record so you can migrate old data without breaking retrieval. This is especially important for episodic memory, which may accumulate months of history.

Enforce Access Control at the Server Level

MCP servers are the natural enforcement point for permissions. A semantic memory server can filter facts by user ID before returning them. Never push raw user-scoped data into the model context without server-side filtering.

Monitor Retrieval Quality

Memory is only useful if the right things come back. Log every retrieval call, the query, the results, and whether the agent used them. Over time you will see patterns — queries that return irrelevant results indicate you need better embeddings, better summaries, or a different chunking strategy.

Handle Context Window Budgets Explicitly

Every memory you pull in consumes tokens. Give the agent a budget and let it prioritize. A practical approach: always include the working memory summary, add the top one or two episodic results, and include semantic facts only when the user references a known entity. Let the model request more with a search_episodes or recall call when needed.

Common Pitfalls

• Storing everything. Dumping raw transcripts into episodic memory creates noise. Summarize first.
• Ignoring staleness. Facts change. Add timestamps and let the agent overwrite or invalidate old values.
• One giant server. Monolithic memory servers become unmaintainable. Split by tier and by domain.
• No evaluation. Without measuring recall quality, you cannot tell if memory is helping. Build a small eval set of multi-turn conversations.
• Forgetting to compact. If working memory grows unchecked, you will hit context limits and degrade quality. Always implement summarization.

Conclusion

Agent memory is not a single feature but an architecture. By splitting memory into working, episodic, semantic, and procedural tiers — and exposing each through dedicated MCP servers — you get a system that is modular, observable, and scalable. MCP handles the plumbing so you can focus on the hard parts: deciding what to remember, when to forget, and how to surface the right context at the right moment. Start with the simple servers in this guide, run them locally, and iterate. As your agent grows, swap in production-grade stores behind the same MCP interfaces, and your architecture will scale with you without a rewrite.