AutoGen Teachability: Making Agents Learn from Interactions

What is Teachability in AutoGen?

Teachability is a built-in capability in Microsoft's AutoGen framework that allows AI agents to learn dynamically from their interactions with users and other agents. When teachability is enabled, an agent can memorize new facts, instructions, preferences, and corrections provided during conversations — then recall and apply that knowledge in future interactions. This transforms agents from static, prompt-bound entities into adaptive systems that grow smarter over time.

At its core, teachability implements a persistent memory layer that sits alongside the agent's standard conversation context. When a user says "remember that I prefer Python over JavaScript" or "please note that our production server runs on port 8080," a teachable agent stores this information and retrieves it when relevant in subsequent exchanges. The agent doesn't simply parrot back memorized facts — it uses them to inform its reasoning, code generation, and decision-making in contextually appropriate ways.

Why Teachability Matters

Without teachability, agents suffer from a fundamental limitation: every interaction starts from scratch. An agent might solve the same problem for the same user dozens of times without ever retaining the user's preferences, environment details, or past solutions. This creates several practical problems:

• Repetitive corrections: Users must repeatedly specify the same preferences (e.g., "use async/await patterns," "target Python 3.11," "my database is PostgreSQL") in every session.
• No institutional knowledge: Agents cannot accumulate project-specific conventions, API keys (stored safely), or architectural decisions across conversations.
• Frustrating user experience: The agent appears forgetful and unreliable, undermining trust in AI-assisted workflows.
• Wasted tokens and context: Repetitive instructions consume precious context windows that could be used for actual problem-solving.

Teachability addresses these issues by giving agents a controlled, secure mechanism to persist knowledge across sessions. This is especially valuable in development environments where agents serve as long-term coding assistants, DevOps helpers, or system administrators that need to remember infrastructure details, coding standards, and team conventions.

How Teachability Works Under the Hood

Teachability in AutoGen is implemented through a combination of prompt augmentation, a persistent memory store, and a specialized assistant agent. Here's the architectural breakdown:

The MemoStore

At the storage level, teachability uses a MemoStore — a persistent database (backed by a JSON file, SQLite, or a custom backend) that stores memory entries. Each entry consists of a natural language description and a timestamp. When a user teaches the agent something, the teachability module extracts the key information and writes a concise memo to this store.

The Teachability Agent

AutoGen wraps a standard assistant agent with a Teachability object that intercepts messages. This wrapper performs two critical functions:

• Memorization: When it detects an instruction or correction from the user (triggered by phrases like "remember that," "note that," "for future reference," or explicit teaching commands), it parses the content and writes a new memo to the store.
• Recall: Before the agent processes a new user message, the teachability module searches the MemoStore for relevant memories based on semantic similarity to the current task. Retrieved memories are prepended to the agent's system prompt as "Remember for this interaction" context.

Integration with the Conversation Loop

Teachability sits transparently in the agent's message processing pipeline. When a user sends a message, the flow is: incoming message → memory recall (search MemoStore) → augment system prompt with relevant memories → agent generates response → check for teachable moments → store new memories if detected → return response to user. This entire cycle happens within the standard AutoGen conversation loop without requiring changes to how users interact with agents.

Getting Started: Enabling Teachability

To enable teachability, you need to import the capability, create a MemoStore, and attach teachability to an existing assistant agent. The minimal setup requires only a few lines of code beyond the standard AutoGen configuration.

Prerequisites

Ensure you have AutoGen installed with the teachability dependencies. The teachability module uses text embeddings for semantic memory retrieval, so you'll need either OpenAI embeddings or a local embedding model.

pip install pyautogen[teachable]

Or with all extras:

pip install pyautogen[all]

Basic Setup Example

The following code creates a teachable assistant agent that remembers user preferences across conversations. The MemoStore persists to a local JSON file so memories survive restarts.

import os
from autogen import AssistantAgent, UserProxyAgent
from autogen.agentchat.contrib.capabilities.teachability import Teachability
from autogen.agentchat.contrib.capabilities.teachability import MemoStore
from autogen import config_list_from_json

# Load your LLM configuration (set environment variables or use a config file)
config_list = config_list_from_json(
    env_or_file="OAI_CONFIG_LIST",
    filter_dict={"model": ["gpt-4"]}
)

# Define the LLM configuration for the assistant
llm_config = {
    "config_list": config_list,
    "timeout": 120,
    "temperature": 0.1,
}

# Create a standard assistant agent
assistant = AssistantAgent(
    name="TeachingAssistant",
    system_message="You are a helpful AI assistant with teachability enabled. 
You can learn and remember information that users teach you.",
    llm_config=llm_config,
)

# Create a user proxy agent (simulates the human user)
user_proxy = UserProxyAgent(
    name="User",
    human_input_mode="NEVER",
    max_consecutive_auto_reply=0,
    code_execution_config={"work_dir": "workspace", "use_docker": False},
)

# Create a MemoStore backed by a JSON file for persistence
memo_store = MemoStore(
    storage_backend="json",
    storage_path="./agent_memories.json"
)

# Instantiate teachability and attach it to the assistant
teachability = Teachability(
    verbosity=1,  # Set to 1 for logging, 0 for silent operation
    reset_db=False,  # Keep existing memories; set True to start fresh
    path_to_db_dir="./teachability_db",
    llm_config={"config_list": config_list, "timeout": 120},
    memo_store=memo_store
)

# Add the teachability capability to the assistant
teachability.add_to_agent(assistant)

print("Teachable assistant is ready. Memories will persist in agent_memories.json")

Configuration Options Explained

The Teachability constructor accepts several important parameters that control how learning behaves:

• verbosity: Controls logging level. Set to 1 during development to see which memories are being retrieved and stored. Set to 0 in production for silent operation.
• reset_db: When True, clears all existing memories on initialization. Useful for testing. Set to False to retain knowledge across sessions.
• path_to_db_dir: Directory where ChromaDB stores embedding vectors for semantic search. This is separate from the MemoStore JSON file.
• llm_config: Configuration for the LLM used to extract memories and determine what to store. This can be a cheaper, faster model than your main assistant.
• memo_store: The storage backend for memo text entries. Can be JSON file-based, SQLite, or a custom implementation.

Practical Code Examples

Example 1: Teaching Basic Preferences

This example demonstrates a complete conversation where the user teaches the agent preferences, then verifies recall in a subsequent interaction.

import autogen
from autogen import AssistantAgent, UserProxyAgent
from autogen.agentchat.contrib.capabilities.teachability import Teachability, MemoStore
from autogen import config_list_from_json

# Setup (abbreviated - use the full setup from the previous example)
config_list = config_list_from_json(env_or_file="OAI_CONFIG_LIST")
llm_config = {"config_list": config_list, "timeout": 120, "temperature": 0.1}

assistant = AssistantAgent(
    name="CodingBuddy",
    system_message="You are a coding assistant. Help the user write code based on their preferences.",
    llm_config=llm_config,
)

user_proxy = UserProxyAgent(
    name="User",
    human_input_mode="NEVER",
    max_consecutive_auto_reply=0,
    code_execution_config={"work_dir": "workspace", "use_docker": False},
)

memo_store = MemoStore(storage_backend="json", storage_path="./coding_memories.json")
teachability = Teachability(
    verbosity=1,
    reset_db=True,
    path_to_db_dir="./teachability_db",
    llm_config={"config_list": config_list, "timeout": 120},
    memo_store=memo_store
)
teachability.add_to_agent(assistant)

# First conversation: teach the agent preferences
print("=== Session 1: Teaching preferences ===")
user_proxy.initiate_chat(
    assistant,
    message="""I have some preferences for how you should write code:

1. Remember that I prefer using async/await patterns over synchronous code for I/O operations.
2. Remember that all Python functions should include type hints.
3. Remember that my project uses FastAPI and Pydantic for data validation.

Now, please write a simple function that fetches data from an API and validates it.""",
    max_turns=5
)

print("\n\n=== Session 2: Testing recall ===")
# In a new session (or same session), the agent should remember the preferences
user_proxy.initiate_chat(
    assistant,
    message="Write a function that retrieves user data from a REST endpoint and validates the response.",
    max_turns=5
)
# The agent should automatically use async/await, include type hints, and use Pydantic
# without the user having to repeat those instructions.

Example 2: Teaching Technical Environment Details

A common real-world use case: teaching an agent about your specific development environment, infrastructure, and project conventions so it can give contextually relevant advice.

# Continuing with the same assistant setup from above...

user_proxy.initiate_chat(
    assistant,
    message="""Please remember these details about my environment:

- Remember that our production database is PostgreSQL 15 running on AWS RDS at endpoint db-prod.internal:5432
- Remember that our Redis cache cluster is at redis-cluster.internal:6379 with password stored in env var REDIS_PASSWORD
- Remember that we use Kubernetes 1.28 for orchestration with namespaces dev, staging, and prod
- Remember that all container images are stored in ECR at 123456789.dkr.ecr.us-east-1.amazonaws.com

Now, I need a Python script that connects to the production database and runs a health check query.""",
    max_turns=5
)

# Later, in a new session...
user_proxy.initiate_chat(
    assistant,
    message="Write a script that clears the Redis cache for the staging environment. Use the connection details you remember.",
    max_turns=5
)
# The agent should recall the Redis endpoint and use the env var for the password,
# and know to target the staging namespace in Kubernetes.

Example 3: Iterative Learning and Corrections

Teachability truly shines when agents receive corrections and refine their understanding over multiple interactions. This example shows how an agent's knowledge evolves.

# Session 1: Initial teaching
user_proxy.initiate_chat(
    assistant,
    message="""For this project, remember these conventions:

- Remember that all error handling should use a custom AppError class that inherits from Exception
- Remember that log messages should use structured logging with key-value pairs, not f-strings

Write a function that reads a JSON config file and handles file-not-found errors.""",
    max_turns=5
)

# The agent generates code using AppError and structured logging...
# User reviews and provides a correction in Session 2:

user_proxy.initiate_chat(
    assistant,
    message="""Update your memory: the AppError class is actually in the module 
`common.exceptions` and is imported as `from common.exceptions import AppError`. 
Also, remember that we use `loguru` for structured logging, not the standard logging module.

Now rewrite the config reader function with these corrections.""",
    max_turns=5
)

# Session 3: Verify the updated knowledge is applied
user_proxy.initiate_chat(
    assistant,
    message="Write a function that reads both a JSON config and a YAML config, 
merging them, with proper error handling for each file type.",
    max_turns=5
)
# The agent should now correctly import AppError from common.exceptions
# and use loguru for structured logging, incorporating both corrections.

Best Practices for Effective Teachability

1. Use Explicit Teaching Language

While teachability can infer some learnings from context, the most reliable approach is to use clear, explicit teaching phrases. The module looks for patterns like "remember that," "note that," "for future reference," "please learn," and similar directives. Phrasing matters:

• Effective: "Remember that our authentication service uses JWT tokens with RS256 algorithm and the public key is at /auth/.well-known/jwks.json"
• Less effective: "By the way, we're using JWT RS256 with the key at the standard JWKS endpoint" (the agent might treat this as casual conversation rather than a teachable moment)

2. Keep Memories Atomic and Specific

Each teachable moment should capture one coherent piece of information. Instead of dumping an entire infrastructure document in one "remember that" statement, break it into discrete, independently useful facts:

# Good: atomic, specific memories
"Remember that the users table in PostgreSQL uses UUID primary keys generated via uuid_generate_v4()"
"Remember that all timestamps in the database are stored in UTC without timezone"
"Remember that the API rate limit is 1000 requests per minute per API key"

# Avoid: overly broad, vague memories
"Remember everything about our database setup including schemas, indexes, and connection pooling"

3. Regularly Review and Prune Memories

Over time, the MemoStore can accumulate outdated or contradictory information. Implement a maintenance strategy:

# Periodically inspect stored memories
memo_store = MemoStore(storage_backend="json", storage_path="./agent_memories.json")
all_memos = memo_store.get_all_memos()
for memo in all_memos:
    print(f"[{memo['timestamp']}] {memo['text']}")

# Manually delete outdated memories (implementation depends on MemoStore backend)
# For JSON backend, you can filter and rewrite:
current_memos = [m for m in all_memos if not is_outdated(m)]
memo_store.replace_all_memos(current_memos)  # Hypothetical method

# Alternatively, use reset_db=True periodically in a controlled way
teachability = Teachability(
    reset_db=True,  # Start fresh, then re-teach current facts
    path_to_db_dir="./teachability_db",
    llm_config={"config_list": config_list, "timeout": 120},
    memo_store=memo_store
)

4. Balance Teaching with the Agent's System Prompt

Teachability supplements — but doesn't replace — the agent's system prompt. Use the system prompt for stable, permanent behavioral instructions (role, tone, safety constraints, fundamental coding philosophy) and use teachability for dynamic, evolving knowledge (project-specific conventions, user preferences, environment details). This separation prevents the system prompt from becoming bloated and keeps memories modular.

5. Test Recall with Deliberate Verification

After teaching an agent important information, verify recall in a separate conversation turn or session. This catches extraction failures early:

# After teaching...
user_proxy.initiate_chat(
    assistant,
    message="I've taught you several things. To verify, please list what you remember about my project's technical stack and preferences. Be specific.",
    max_turns=3
)
# Review the agent's response to confirm all key facts were captured correctly

6. Use a Dedicated LLM for Memory Extraction

The teachability module uses an LLM to parse teachable moments and extract concise memories. This doesn't need to be your most powerful model. Configure a faster, cheaper model for this task to reduce latency and cost:

# Use a separate, lighter config for memory extraction
memory_llm_config = {
    "config_list": config_list_from_json(
        env_or_file="OAI_CONFIG_LIST",
        filter_dict={"model": ["gpt-3.5-turbo"]}  # Cheaper, faster model
    ),
    "timeout": 60,
    "temperature": 0.0,  # Deterministic extraction
}

teachability = Teachability(
    verbosity=1,
    reset_db=False,
    path_to_db_dir="./teachability_db",
    llm_config=memory_llm_config,  # Separate LLM for teachability operations
    memo_store=memo_store
)

Advanced Patterns: Conditional Memorization

For production deployments, you may want finer control over what gets memorized. The teachability module supports customization through its recall and memorization thresholds, and you can subclass MemoStore for custom filtering logic.

Adjusting Recall Relevance Thresholds

The semantic similarity threshold determines how closely a memory must match the current task to be recalled. Lower thresholds retrieve more memories (potentially including irrelevant ones); higher thresholds are more conservative.

teachability = Teachability(
    verbosity=1,
    reset_db=False,
    path_to_db_dir="./teachability_db",
    llm_config=memory_llm_config,
    memo_store=memo_store,
    recall_threshold=0.65,  # Default is around 0.5-0.6; increase for stricter matching
    max_recalled_memories=5,  # Limit how many memories are prepended to avoid context overflow
)

Custom MemoStore with Filtering

You can implement a custom MemoStore that filters or organizes memories by category, user, or project:

from autogen.agentchat.contrib.capabilities.teachability import MemoStore
import json
import os
from datetime import datetime

class CategorizedMemoStore(MemoStore):
    """A MemoStore that organizes memories by user-defined categories."""
    
    def __init__(self, storage_path="./categorized_memories.json"):
        super().__init__(storage_backend="json", storage_path=storage_path)
        self._ensure_storage()
    
    def _ensure_storage(self):
        if not os.path.exists(self.storage_path):
            with open(self.storage_path, "w") as f:
                json.dump({"memos": [], "categories": {}}, f)
    
    def add_memo(self, text, category="general", metadata=None):
        """Add a memo with an associated category for better organization."""
        memo_entry = {
            "text": text,
            "category": category,
            "timestamp": datetime.utcnow().isoformat(),
            "metadata": metadata or {}
        }
        # Store in the JSON backend
        data = self._load_data()
        data["memos"].append(memo_entry)
        if category not in data["categories"]:
            data["categories"][category] = []
        data["categories"][category].append(len(data["memos"]) - 1)  # Index reference
        self._save_data(data)
        return memo_entry
    
    def get_memos_by_category(self, category):
        """Retrieve all memos in a specific category."""
        data = self._load_data()
        indices = data["categories"].get(category, [])
        return [data["memos"][i] for i in indices]
    
    def _load_data(self):
        with open(self.storage_path, "r") as f:
            return json.load(f)
    
    def _save_data(self, data):
        with open(self.storage_path, "w") as f:
            json.dump(data, f, indent=2)

# Usage
categorized_store = CategorizedMemoStore("./project_memories.json")

# Teach with categories
user_proxy.initiate_chat(
    assistant,
    message="""Remember that [category: infrastructure] our load balancer timeout is 30 seconds.
Remember that [category: security] all internal APIs require mTLS certificates.
Remember that [category: python-style] we use black for code formatting with line length 100.""",
    max_turns=3
)

Common Pitfalls and How to Avoid Them

Pitfall 1: Context Window Overflow

Problem: As the MemoStore grows, too many recalled memories consume the agent's context window, leaving less room for the actual task. The agent may receive 15+ memories for a complex query, crowding out code and reasoning space.

Solution: Set max_recalled_memories to a reasonable limit (3-7) and increase the recall threshold to only retrieve highly relevant memories. Additionally, periodically curate the MemoStore to remove obsolete entries.

teachability = Teachability(
    # ... other config ...
    recall_threshold=0.75,      # Only highly relevant matches
    max_recalled_memories=4,    # Hard limit on recalled memories
)

Pitfall 2: Memory Contamination Across Projects

Problem: An agent used for multiple projects accumulates memories from all of them, causing it to apply Project A's conventions when working on Project B.

Solution: Use separate MemoStore instances (separate JSON files or database tables) for different projects or contexts. Switch between them based on the active project.

# Create project-specific memo stores
project_a_store = MemoStore(storage_backend="json", storage_path="./project_a_memories.json")
project_b_store = MemoStore(storage_backend="json", storage_path="./project_b_memories.json")

# Attach the appropriate store based on context
def get_teachable_assistant_for_project(project_name):
    store_path = f"./memories_{project_name}.json"
    memo_store = MemoStore(storage_backend="json", storage_path=store_path)
    teachability = Teachability(
        verbosity=0,
        reset_db=False,
        path_to_db_dir=f"./teachability_db_{project_name}",
        llm_config=memory_llm_config,
        memo_store=memo_store
    )
    assistant = AssistantAgent(
        name=f"Assistant_{project_name}",
        system_message=f"You are assisting with project {project_name}.",
        llm_config=llm_config,
    )
    teachability.add_to_agent(assistant)
    return assistant

Pitfall 3: Conflicting or Stale Memories

Problem: A user teaches a fact, later corrects it, but the old memory persists alongside the new one. The agent may retrieve both and become confused about which is current.

Solution: When correcting information, explicitly instruct the agent to update or replace the previous memory. You can also implement versioning in a custom MemoStore.

# Explicit update pattern
user_proxy.initiate_chat(
    assistant,
    message="""Update your memory: replace the previous information about our database endpoint.
The correct, current information is: our production database is now PostgreSQL 16 on RDS 
at endpoint db-prod-v2.internal:5432. The old endpoint db-prod.internal is deprecated.""",
    max_turns=3
)

Pitfall 4: Sensitive Information Leakage

Problem: Users might inadvertently teach the agent sensitive information (API keys, passwords, proprietary algorithms) that gets stored in plain text in the MemoStore JSON file.

Solution: Never teach agents actual secrets. Instead, teach them where to find secrets (e.g., "Remember that API keys are stored in environment variables with prefix PROD_"). Implement a pre-processing filter that rejects potential secrets before storage.

# Simple secrets filter for the MemoStore
import re

class SafeMemoStore(MemoStore):
    SENSITIVE_PATTERNS = [
        r'(?i)(api[_-]?key|secret|password|token|credential)\s*[:=]\s*\S+',
        r'(?i)Bearer\s+\S+',
        r'(?i)-----BEGIN\s+(RSA|EC|DSA|OPENSSH)\s+PRIVATE\s+KEY-----',
    ]
    
    def add_memo(self, text, **kwargs):
        for pattern in self.SENSITIVE_PATTERNS:
            if re.search(pattern, text):
                raise ValueError(f"Memo contains potential sensitive information: {text[:50]}...")
        return super().add_memo(text, **kwargs)

Conclusion

Teachability transforms AutoGen agents from stateless responders into persistent, learning collaborators that accumulate knowledge across interactions. By combining a semantic memory store with intelligent recall mechanisms, teachable agents reduce repetition, adapt to user preferences, and build project-specific expertise over time. The key to successful implementation lies in using explicit teaching language, keeping memories atomic, managing context window limits, and maintaining separate memory stores for different contexts. When deployed thoughtfully — with attention to memory curation, security filtering, and threshold tuning — teachability creates a dramatically more productive and natural interaction experience where agents genuinely learn and improve with each conversation.