CrewAI Memory Systems: Short-Term and Long-Term Agent Memory

Understanding Memory Systems in CrewAI

Memory is one of the most critical components in building effective AI agent teams. In CrewAI, memory systems allow agents to retain context, learn from past interactions, and make informed decisions across both immediate tasks and extended workflows. CrewAI provides two primary memory layers: Short-Term Memory and Long-Term Memory. Each serves a distinct purpose and can be configured independently to suit your application's needs.

What Are CrewAI Memory Systems?

CrewAI's memory architecture mirrors human cognitive memory models. Short-term memory handles immediate, task-scoped information—things the agent needs to know right now to complete a current job. Long-term memory persists beyond a single execution cycle, storing insights, facts, and learnings that can be recalled in future runs, even days or weeks later.

The framework offers three concrete memory implementations out of the box:

• Short-Term Memory (Context Window) — In-memory storage that lives for the duration of a single Crew execution. It holds the current conversation, tool outputs, and intermediate reasoning steps.
• Long-Term Memory (Persistent) — Backed by a vector store and embedding model, this memory persists across executions. It stores task results, learned facts, and agent reflections.
• Entity Memory — A specialized long-term memory variant that tracks specific entities (people, organizations, concepts) mentioned across conversations, enabling rich entity-aware reasoning.

Why Memory Matters in Agentic Systems

Without memory, every agent interaction starts from a blank slate. The agent has no idea what happened before, what was learned, or what decisions were made. This leads to:

• Repetitive mistakes — Agents re-evaluate problems they've already solved
• Lost context — Multi-step workflows break because intermediate results are forgotten
• Poor user experience — Users must repeatedly provide the same information
• No learning curve — The system never improves from past experience

Memory transforms a reactive agent into a context-aware, learning system. Short-term memory enables coherent multi-turn reasoning within a single task. Long-term memory allows the entire crew to evolve, accumulating organizational knowledge that compounds over time.

Short-Term Memory: The Agent's Working Context

How Short-Term Memory Works

Short-term memory in CrewAI is automatically managed through the agent's context window. When an agent executes a task, CrewAI feeds it a prompt that includes the current task description, the agent's role and backstory, available tools, and—crucially—the conversation history and intermediate results from the current execution. This context is ephemeral: once the Crew's kickoff() or kickoff_async() completes, short-term memory is cleared.

Under the hood, short-term memory is implemented via CrewAI's internal Context management. Every task output, tool call result, and delegation response is appended to a running transcript that subsequent agents can access. This is not a vector database—it's raw text stored in memory during the active session.

Configuring Short-Term Memory

Short-term memory is enabled by default and requires no explicit setup. However, you can control its behavior through the memory parameter on the Crew and the context parameter on individual tasks. Here's a basic example demonstrating how short-term context flows between tasks:

from crewai import Crew, Agent, Task, Process

# Define agents
researcher = Agent(
    role="Market Researcher",
    goal="Gather and analyze market data for {topic}",
    backstory="You are an expert at finding market trends and insights.",
    verbose=True,
    allow_delegation=False,
)

analyst = Agent(
    role="Financial Analyst",
    goal="Analyze the research data and produce a financial forecast",
    backstory="You excel at turning raw data into actionable financial projections.",
    verbose=True,
    allow_delegation=False,
)

# Task 1: Research - its output goes into short-term memory automatically
research_task = Task(
    description="Research the current market landscape for {topic}. "
                "Identify key players, trends, and market size.",
    expected_output="A comprehensive market research report with key metrics.",
    agent=researcher,
)

# Task 2: Analysis - uses short-term memory to access the research output
analysis_task = Task(
    description="Using the market research provided, create a financial "
                "forecast for the next 12 months. Reference specific numbers "
                "from the research.",
    expected_output="A financial forecast document with revenue projections.",
    agent=analyst,
    context=[research_task],  # Explicitly links to research task's output
)

# Assemble the crew
crew = Crew(
    agents=[researcher, analyst],
    tasks=[research_task, analysis_task],
    process=Process.sequential,
    verbose=True,
)

# Execute - short-term memory flows research -> analysis
result = crew.kickoff(inputs={"topic": "Electric Vehicle Batteries"})
print(result)

In this example, the analyst agent automatically receives the researcher's output in its context window. The context=[research_task] parameter ensures the task output is explicitly injected into the prompt. Even without the context parameter, sequential tasks naturally pass outputs forward—but explicit context gives you fine-grained control.

Accessing Short-Term Memory During Execution

Agents can introspect their own short-term memory during a task using the Task object's stored context. You can also inject custom short-term data via the inputs dictionary when calling kickoff(). Here's an advanced pattern where an agent dynamically pulls from short-term context:

from crewai import Agent, Task, Crew

# Agent with a custom tool that accesses short-term context
summarizer = Agent(
    role="Conversation Summarizer",
    goal="Summarize all previous interactions and extract key decisions",
    backstory="You are meticulous about tracking decisions and action items.",
    verbose=True,
    tools=[],  # Custom tools would go here
)

# A task that explicitly references short-term memory
summary_task = Task(
    description="Review the entire conversation history so far. "
                "Extract: 1) All decisions made, 2) Open questions, "
                "3) Action items with owners. Format as a structured memo.",
    expected_output="A structured memo with decisions, questions, and action items.",
    agent=summarizer,
)

crew = Crew(
    agents=[summarizer],
    tasks=[summary_task],
    process=Process.sequential,
    memory=True,  # Enables memory (default True, but explicit here)
)

result = crew.kickoff()
print(result)

Short-Term Memory Limitations

Short-term memory has practical constraints you must design around:

• Context window limits — Most LLMs have token limits (e.g., 128K for GPT-4o, 200K for Claude). Long conversations or large tool outputs may exceed these limits, causing truncation.
• No persistence — Once execution ends, all short-term context is lost. Restarting the crew means starting from scratch.
• No semantic retrieval — Short-term memory is chronological, not searchable by similarity. Agents must linearly process context.

These limitations are exactly why long-term memory exists.

Long-Term Memory: Persistent Knowledge Across Executions

How Long-Term Memory Works

Long-term memory in CrewAI persists data across multiple kickoff() calls using a combination of embedding models and vector stores. When long-term memory is enabled, CrewAI automatically:

1. Embeds task outputs, agent reflections, and learned facts into vector representations
2. Stores these embeddings in a configurable vector database (in-memory by default, or external like ChromaDB, Pinecone, etc.)
3. Retrieves relevant memories before each new task execution based on semantic similarity
4. Injects retrieved memories into the agent's context window alongside the current task

This creates a self-improving loop: the more you use your crew, the smarter it becomes as it accumulates and retrieves relevant past experiences.

Enabling Long-Term Memory

Long-term memory is enabled by setting memory=True on the Crew and configuring a memory provider. The simplest setup uses CrewAI's default in-memory vector store with OpenAI embeddings:

from crewai import Crew, Agent, Task, Process
from crewai.memory import LongTermMemory
from crewai.memory.storage import LTMSQLiteStorage
from crewai.memory.storage.kickoff_task_output_storage import (
    KickoffTaskOutputStorage,
)

# Configure long-term memory with SQLite storage
long_term_memory = LongTermMemory(
    storage=LTMSQLiteStorage(
        db_path="./my_crew_memory.db"  # Persistent storage file
    )
)

# Create agents as usual
coder = Agent(
    role="Senior Software Engineer",
    goal="Write clean, well-tested Python code based on requirements",
    backstory="You have 10 years of experience and remember every codebase you've worked on.",
    verbose=True,
)

reviewer = Agent(
    role="Code Reviewer",
    goal="Review code for bugs, style, and performance issues",
    backstory="You are a meticulous reviewer with deep knowledge of best practices.",
    verbose=True,
)

coding_task = Task(
    description="Write a Python FastAPI endpoint that handles user authentication "
                "with JWT tokens. Include proper error handling.",
    expected_output="Complete FastAPI route code with JWT authentication logic.",
    agent=coder,
)

review_task = Task(
    description="Review the authentication code produced. Check for security issues, "
                "proper token validation, and adherence to FastAPI best practices.",
    expected_output="A detailed code review with specific improvement suggestions.",
    agent=reviewer,
    context=[coding_task],
)

crew = Crew(
    agents=[coder, reviewer],
    tasks=[coding_task, review_task],
    process=Process.sequential,
    memory=True,              # Enable long-term memory
    long_term_memory=long_term_memory,  # Pass the configured memory instance
    verbose=True,
)

# First execution - memories are stored
result_1 = crew.kickoff()
print("First run complete. Memories stored.")

# Second execution - relevant memories are retrieved
result_2 = crew.kickoff()
print("Second run complete. Previous memories were used.")

After the first run, task outputs and agent reflections are embedded and stored in my_crew_memory.db. On the second run, before each task begins, CrewAI retrieves semantically similar memories and prepends them to the agent's prompt, giving the agent "knowledge" of what happened previously.

Using External Vector Stores

For production deployments, you'll want to use a dedicated vector database. CrewAI supports several backends. Here's how to configure ChromaDB as the long-term memory store:

import os
from crewai import Crew, Agent, Task
from crewai.memory import LongTermMemory
from crewai.memory.storage import ChromaDBStorage
from chromadb.config import Settings

# Configure ChromaDB persistent storage
chroma_storage = ChromaDBStorage(
    chroma_settings=Settings(
        chroma_server_host="localhost",
        chroma_server_http_port=8000,
        chroma_persistent_path="./chroma_crew_data",
    ),
    collection_name="crew_memories",
)

long_term_memory = LongTermMemory(
    storage=chroma_storage,
    embedding_model="openai",  # Uses OPENAI_API_KEY from environment
)

# Agent and task definitions...
support_agent = Agent(
    role="Customer Support Specialist",
    goal="Resolve customer issues by referencing past solutions and knowledge base",
    backstory="You're an empathetic support agent who remembers every customer interaction.",
    verbose=True,
)

support_task = Task(
    description="A customer reports that their API key is not working. "
                "Diagnose the issue and provide a solution.",
    expected_output="A clear diagnostic and resolution steps for the customer.",
    agent=support_agent,
)

crew = Crew(
    agents=[support_agent],
    tasks=[support_task],
    memory=True,
    long_term_memory=long_term_memory,
    verbose=True,
)

# Run multiple times to build up memory
for i in range(3):
    print(f"\n--- Interaction {i+1} ---")
    result = crew.kickoff()
    print(result)

Memory Configuration Options

CrewAI's long-term memory system offers several configuration parameters to fine-tune behavior:

from crewai.memory import LongTermMemory
from crewai.memory.storage import LTMSQLiteStorage

long_term_memory = LongTermMemory(
    storage=LTMSQLiteStorage(
        db_path="./crew_memory.db",
    ),
    # Number of memories to retrieve per query (default: 3)
    memory_limit=5,
    # Minimum similarity score threshold (0.0 to 1.0)
    similarity_threshold=0.75,
    # Whether to store agent "learnings" explicitly
    store_learnings=True,
    # Whether to store raw task outputs
    store_task_outputs=True,
    # Custom embedding model (default: "openai")
    embedding_model="openai",
    # Alternatively, use a custom embedding function
    # embedding_function=my_custom_embedding_function,
)

crew = Crew(
    agents=[...],
    tasks=[...],
    memory=True,
    long_term_memory=long_term_memory,
)

Adjusting similarity_threshold is particularly important. Set it higher (0.8+) for precision—only highly relevant memories are retrieved. Set it lower (0.5–0.6) for recall—more memories are retrieved but some may be less relevant.

Customizing the Embedding Model

By default, CrewAI uses OpenAI's text-embedding-ada-002 model. You can switch to any embedding provider:

# Using OpenAI (default)
long_term_memory = LongTermMemory(
    storage=my_storage,
    embedding_model="openai",
)

# Using a custom embedding function
from sentence_transformers import SentenceTransformer

local_embedder = SentenceTransformer("all-MiniLM-L6-v2")

def custom_embed(text: str) -> list[float]:
    """Custom embedding function compatible with CrewAI."""
    return local_embedder.encode(text).tolist()

long_term_memory = LongTermMemory(
    storage=my_storage,
    embedding_function=custom_embed,
    # embedding_model is ignored when embedding_function is provided
)

This gives you complete control over the embedding pipeline, allowing you to use local models, HuggingFace embeddings, or any other provider.

Entity Memory: Remembering Specific Things

What Entity Memory Is

Entity memory is a specialized long-term memory variant that tracks specific entities—people, organizations, products, locations—mentioned across interactions. It builds a knowledge graph-like structure where each entity accumulates attributes and facts over time. For example, if a customer named "Alice Johnson" appears in multiple support tickets, entity memory consolidates everything known about Alice across all interactions.

Setting Up Entity Memory

from crewai import Crew, Agent, Task
from crewai.memory import LongTermMemory, EntityMemory
from crewai.memory.storage import LTMSQLiteStorage

# Shared storage can back multiple memory types
storage = LTMSQLiteStorage(db_path="./full_crew_memory.db")

long_term_memory = LongTermMemory(storage=storage)
entity_memory = EntityMemory(storage=storage)

# Agent that leverages entity memory
crm_agent = Agent(
    role="CRM Specialist",
    goal="Maintain detailed records of every customer interaction and preference",
    backstory="You know every customer personally and remember all their details.",
    verbose=True,
)

customer_task = Task(
    description="Alice Johnson (alice@example.com) called about upgrading "
                "her subscription from Basic to Premium. She mentioned she "
                "uses the service primarily for data analytics and needs "
                "the advanced reporting features. Process this upgrade.",
    expected_output="Confirmation of upgrade and personalized follow-up recommendations.",
    agent=crm_agent,
)

crew = Crew(
    agents=[crm_agent],
    tasks=[customer_task],
    memory=True,
    long_term_memory=long_term_memory,
    entity_memory=entity_memory,
    verbose=True,
)

# First interaction - learns about Alice
crew.kickoff()

# Later interaction - entity memory recalls Alice
followup_task = Task(
    description="Alice Johnson just emailed asking about the advanced reporting "
                "features. Based on what you know about her, provide a tailored response.",
    expected_output="Personalized response referencing her known preferences.",
    agent=crm_agent,
)

crew2 = Crew(
    agents=[crm_agent],
    tasks=[followup_task],
    memory=True,
    long_term_memory=long_term_memory,
    entity_memory=entity_memory,
    verbose=True,
)

crew2.kickoff()  # Agent remembers Alice from the first interaction

Best Practices for CrewAI Memory Systems

1. Start Simple, Then Scale

Begin with default short-term memory only. Validate that your crew works correctly for single-run scenarios. Once task flows are stable, enable long-term memory with the default in-memory store. Only move to external vector databases (ChromaDB, Pinecone, Weaviate) when you need persistence across application restarts or when scaling to many concurrent crews.

2. Tune Similarity Thresholds

The similarity_threshold parameter dramatically affects memory quality. Start at 0.7 and observe whether agents receive relevant context. If they seem to miss important past information, lower the threshold to 0.5–0.6. If they're getting flooded with irrelevant memories, raise it to 0.8–0.85. Run multiple trials to find the sweet spot for your use case.

3. Clean Memory Periodically

Over time, long-term memory accumulates noise—outdated facts, failed approaches, or irrelevant tangents. Implement a memory maintenance strategy:

• Scheduled pruning — Periodically clear memories older than a certain age
• Relevance scoring — Track how often a memory is retrieved; archive those rarely accessed
• Explicit forgetting — Build a "forget" task that lets agents mark memories as obsolete

# Example: pruning old memories (conceptual pattern)
from datetime import datetime, timedelta

def prune_old_memories(storage, max_age_days=30):
    """Remove memories older than max_age_days."""
    cutoff = datetime.now() - timedelta(days=max_age_days)
    # Implementation depends on your storage backend
    # For SQLite: DELETE FROM memories WHERE created_at < cutoff
    # For ChromaDB: filter by metadata timestamp
    storage.delete_older_than(cutoff)
    print(f"Pruned memories older than {max_age_days} days.")

# Call periodically
prune_old_memories(long_term_memory.storage, max_age_days=90)

4. Balance Memory with Context Window Limits

Retrieved long-term memories consume precious context window tokens. Set memory_limit thoughtfully. If your agent already has a large task description and tool outputs, limit retrieved memories to 2–3 to avoid overflow. Monitor token usage and adjust accordingly.

5. Use Explicit Context Links

Don't rely solely on automatic memory retrieval. Use the context parameter on tasks to explicitly wire dependencies. This ensures critical information is always present, while long-term memory provides supplementary context:

task_3 = Task(
    description="Synthesize findings into an executive summary.",
    expected_output="A one-page executive summary.",
    agent=synthesizer,
    context=[task_1, task_2],  # Explicit dependencies (always included)
    # Long-term memory adds supplementary context automatically
)

6. Separate Memory Stores for Different Crews

If you run multiple crews handling distinct domains (e.g., a sales crew and a legal crew), give them separate memory stores. Cross-contamination between domains leads to confusing, irrelevant memory retrieval:

# Sales crew gets its own memory
sales_memory = LongTermMemory(
    storage=LTMSQLiteStorage(db_path="./sales_memory.db")
)

# Legal crew gets separate memory
legal_memory = LongTermMemory(
    storage=LTMSQLiteStorage(db_path="./legal_memory.db")
)

sales_crew = Crew(..., long_term_memory=sales_memory)
legal_crew = Crew(..., long_term_memory=legal_memory)

7. Log and Monitor Memory Operations

Enable verbose logging during development to understand exactly what memories are being retrieved and injected. This helps debug when agents produce unexpected outputs:

crew = Crew(
    agents=[...],
    tasks=[...],
    memory=True,
    long_term_memory=long_term_memory,
    verbose=True,  # Shows memory retrieval in console output
    # For even more detail, set logging level
    # logger=logging.getLogger("crewai.memory"),
)

8. Handle Memory in Async and Streaming Modes

Long-term memory works identically with kickoff_async() and streaming callbacks. Memories are stored and retrieved per task execution, regardless of synchronous or asynchronous operation:

import asyncio

async def run_async_crew_with_memory():
    crew = Crew(
        agents=[my_agent],
        tasks=[my_task],
        memory=True,
        long_term_memory=long_term_memory,
    )
    
    # Async execution still uses memory
    result = await crew.kickoff_async()
    return result

# Streaming with memory
def streaming_callback(chunk: str):
    print(chunk, end="", flush=True)

crew = Crew(
    agents=[my_agent],
    tasks=[my_task],
    memory=True,
    long_term_memory=long_term_memory,
    verbose=True,
)

result = crew.kickoff(callbacks=[streaming_callback])

Debugging Memory Issues

When memory isn't working as expected, systematically check these areas:

• Embedding model availability — If using OpenAI embeddings, ensure OPENAI_API_KEY is set. For local models, verify the model is loaded correctly.
• Storage connection — For external stores like ChromaDB, confirm the service is running and accessible from your application.
• Similarity threshold — If no memories are retrieved, the threshold may be too high. Temporarily set it to 0.0 to verify memories exist.
• Memory limit — If memory_limit=0 or is set too low, no memories will be injected even if matches exist.
• Task isolation — Ensure tasks are properly linked with context where needed; memory supplements but doesn't replace explicit task dependencies.

Conclusion

CrewAI's memory systems transform your agent teams from stateless workers into persistent, learning organizations. Short-term memory provides the working context agents need to collaborate effectively within a single execution, while long-term memory accumulates knowledge across executions, enabling your crew to grow smarter with every run. Entity memory adds another dimension by tracking specific people, organizations, and concepts across interactions. By configuring storage backends, tuning similarity thresholds, and following best practices around memory hygiene and domain separation, you can build CrewAI systems that deliver increasingly intelligent results over time. Start with the defaults, observe how your agents behave, and incrementally tune memory parameters to match your specific use case. The investment in understanding and configuring memory pays dividends in agent performance and user satisfaction.