Introduction to Context Window Optimization with MCP
The Model Context Protocol (MCP) is an open standard introduced to standardize how AI applications connect to external data sources, tools, and services. As large language models (LLMs) continue to grow in capability, one of the most persistent bottlenecks developers face is the context window — the limited amount of text a model can process in a single request. Context window optimization with MCP is the practice of strategically managing what information enters that window, when it enters, and how it is structured to maximize model performance while minimizing token waste.
This guide walks through the theory, architecture, and practical implementation of context window optimization using MCP. Whether you are building coding assistants, retrieval-augmented generation (RAG) systems, or autonomous agents, these techniques will help you build more efficient, cost-effective, and capable AI applications.
What Is the Context Window Problem?
Every LLM has a maximum context length measured in tokens. For example, a model might support 128,000 tokens of input. While that sounds large, it fills up quickly when you account for:
- System prompts and instructions
- Conversation history
- Retrieved documents and knowledge base snippets
- Tool definitions and schemas
- Function call results and intermediate reasoning
When the context window is poorly managed, several problems emerge. The model may truncate important information, latency increases as the input grows, API costs rise proportionally to token usage, and the model's reasoning quality can degrade — a phenomenon sometimes called the "lost in the middle" effect, where information placed in the center of long contexts is more likely to be ignored.
How MCP Changes the Equation
MCP addresses this by providing a structured protocol for just-in-time context injection. Instead of stuffing everything the model might need into the prompt upfront, MCP lets you expose resources, tools, and prompts as discrete endpoints that the model can query on demand. This shifts the paradigm from "push everything in" to "pull what you need."
An MCP server can expose three primitive types:
- Resources — Read-only data sources the model can access, such as files, database records, or API responses.
- Tools — Executable functions the model can call, such as running a search query or executing code.
- Prompts — Pre-defined prompt templates that can be dynamically populated and inserted into context.
Why Context Window Optimization Matters
Optimizing the context window is not just about saving tokens. It has direct implications for the quality, speed, and reliability of your AI application. Here are the key reasons it matters:
1. Cost Efficiency
Most LLM providers charge per token. If you are sending 50,000 tokens when 5,000 would suffice, you are paying ten times more than necessary. Over millions of requests, this compounds dramatically. MCP-based optimization lets you keep the baseline context lean and only expand it when the model determines additional data is needed.
2. Latency Reduction
Time to first token and overall generation time both scale with input length. A leaner context window means faster responses, which is critical for interactive applications like coding assistants or chat interfaces.
3. Model Performance
Research has shown that models perform better when context is relevant and well-organized. Irrelevant information acts as noise, increasing the chance of hallucinations and reducing the model's ability to focus on the task at hand. By curating context carefully, you improve answer quality.
4. Scalability
As your application grows — more tools, more data sources, more conversation history — the pressure on the context window increases exponentially. Without a systematic optimization strategy, you will hit hard limits. MCP provides the architectural foundation to scale gracefully.
How MCP Works: Architecture Overview
MCP follows a client-server architecture. The MCP client runs inside your AI application (or is integrated into the LLM host environment). The MCP server exposes capabilities that the client can discover and invoke. Communication happens over a standardized JSON-RPC protocol, typically over stdio or HTTP with Server-Sent Events (SSE).
The flow looks like this:
- The client connects to one or more MCP servers.
- The client discovers available tools, resources, and prompts through the protocol's discovery mechanism.
- Only lightweight metadata (tool names, descriptions, schemas) is included in the initial context sent to the LLM.
- The LLM decides which tools to call or which resources to read based on the task.
- The client fetches the requested data from the MCP server and injects it into the context.
- The LLM uses the newly available information to continue reasoning or generate a response.
This lazy-loading approach is the core of context window optimization with MCP. Instead of pre-loading megabytes of documentation, the model pulls specific sections only when needed.
Setting Up an MCP Server
Let's build a practical example. We will create an MCP server that exposes a documentation knowledge base as a resource, allowing the model to fetch specific documents on demand rather than loading everything at once.
First, install the MCP Python SDK:
pip install mcpNow, create a basic MCP server:
from mcp.server import Server
from mcp.server.stdio import stdio_server
from mcp.types import Resource, Tool, TextContent
import json
import mcp.server.stdio
# Simulated knowledge base
KNOWLEDGE_BASE = {
"api-reference": "The API exposes endpoints at /v1/users, /v1/orders, and /v1/products. Authentication uses Bearer tokens.",
"getting-started": "To get started, install the SDK with pip install mypackage, then initialize the client with your API key.",
"error-handling": "Errors return JSON with an 'error' field containing 'code' and 'message'. Common codes: 400, 401, 403, 404, 500.",
"rate-limiting": "Rate limits are 100 requests per minute for free tier and 1000 per minute for pro tier. Headers include X-RateLimit-Remaining.",
}
server = Server("docs-server")
@server.list_resources()
async def list_resources() -> list[Resource]:
return [
Resource(
uri=f"docs://knowledge-base/{doc_id}",
name=doc_id,
description=f"Documentation article: {doc_id}",
mimeType="text/plain",
)
for doc_id in KNOWLEDGE_BASE.keys()
]
@server.read_resource()
async def read_resource(uri: str) -> str:
# Parse the URI to extract the document ID
doc_id = uri.replace("docs://knowledge-base/", "")
if doc_id in KNOWLEDGE_BASE:
return KNOWLEDGE_BASE[doc_id]
return f"Document '{doc_id}' not found."
@server.list_tools()
async def list_tools() -> list[Tool]:
return [
Tool(
name="search_docs",
description="Search the documentation knowledge base by keyword. Returns matching document IDs and snippets.",
inputSchema={
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "The search keyword or phrase to look for in the documentation."
}
},
"required": ["query"],
},
)
]
@server.call_tool()
async def call_tool(name: str, arguments: dict) -> list[TextContent]:
if name == "search_docs":
query = arguments.get("query", "").lower()
results = []
for doc_id, content in KNOWLEDGE_BASE.items():
if query in content.lower() or query in doc_id.lower():
snippet = content[:150] + "..." if len(content) > 150 else content
results.append(f"[{doc_id}]: {snippet}")
if not results:
return [TextContent(type="text", text="No matching documents found.")]
return [TextContent(type="text", text="\n\n".join(results))]
return [TextContent(type="text", text=f"Unknown tool: {name}")]
async def main():
async with stdio_server() as (read_stream, write_stream):
await server.run(read_stream, write_stream, server.create_initialization_options())
if __name__ == "__main__":
import asyncio
asyncio.run(main())
This server exposes four documentation articles as resources and a search_docs tool. The model can first search for relevant documents, then read only the ones it needs — keeping the context window minimal at every step.
Implementing Context Window Optimization Strategies
Now that we have an MCP server, let's explore the key strategies for optimizing the context window on the client side.
Strategy 1: Lazy Resource Loading
The most fundamental optimization is to never load resources until the model requests them. Instead of dumping all documentation into the system prompt, expose them as MCP resources and let the model decide what to read.
import anthropic
from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client
async def run_optimized_conversation():
# Connect to the MCP server
server_params = StdioServerParameters(
command="python",
args=["docs_server.py"],
)
async with stdio_client(server_params) as (read, write):
async with ClientSession(read, write) as session:
await session.initialize()
# Discover available tools and resources
tools_result = await session.list_tools()
resources_result = await session.list_resources()
# Build a compact tool description for the LLM
# Only include tool schemas, NOT the actual resource content
tool_definitions = []
for tool in tools_result.tools:
tool_definitions.append({
"name": tool.name,
"description": tool.description,
"input_schema": tool.inputSchema,
})
# Build a compact resource index — just names and descriptions
resource_index = "\n".join([
f"- {r.name}: {r.description}" for r in resources_result.resources
])
system_prompt = f"""You are a helpful documentation assistant.
The following documentation resources are available. Use the search_docs tool
to find relevant documents, then read specific resources only when needed.
Available resources:
{resource_index}
Always search before reading. Never assume document contents."""
client = anthropic.Anthropic()
user_message = "How do I handle rate limiting with this API?"
response = client.messages.create(
model="claude-sonnet-4-20250514",
max_tokens=1024,
system=system_prompt,
tools=tool_definitions,
messages=[{"role": "user", "content": user_message}],
)
# Handle tool calls iteratively
while response.stop_reason == "tool_use":
tool_calls = [b for b in response.content if b.type == "tool_use"]
tool_results = []
for tc in tool_calls:
if tc.name == "search_docs":
result = await session.call_tool("search_docs", tc.input)
tool_results.append({
"type": "tool_result",
"tool_use_id": tc.id,
"content": result.content[0].text,
})
elif tc.name == "read_resource":
# The model can also request to read a full resource
uri = tc.input.get("uri")
content = await session.read_resource(uri)
tool_results.append({
"type": "tool_result",
"tool_use_id": tc.id,
"content": content,
})
# Continue the conversation with tool results
response = client.messages.create(
model="claude-sonnet-4-20250514",
max_tokens=1024,
system=system_prompt,
tools=tool_definitions,
messages=[
{"role": "user", "content": user_message},
{"role": "assistant", "content": response.content},
{"role": "user", "content": tool_results},
],
)
# Print final response
for block in response.content:
if hasattr(block, "text"):
print(block.text)
Notice that the system prompt contains only a compact index of available resources — not their full contents. The model uses the search tool to find what is relevant, then reads only the specific document it needs. This keeps the context window small throughout the interaction.
Strategy 2: Context Summarization and Compaction
In multi-turn conversations, history grows rapidly. A common optimization is to periodically summarize older conversation turns and replace them with a compact summary. MCP can help by exposing a summarization tool or by managing a sliding window of context.
class ContextManager:
def __init__(self, max_history_tokens: int = 8000, summary_threshold: int = 6000):
self.max_history_tokens = max_history_tokens
self.summary_threshold = summary_threshold
self.messages = []
self.summary = ""
def estimate_tokens(self, text: str) -> int:
# Rough estimate: ~4 characters per token
return len(text) // 4
def add_message(self, role: str, content: str):
self.messages.append({"role": role, "content": content})
self._maybe_compact()
def _maybe_compact(self):
total_tokens = sum(
self.estimate_tokens(m["content"]) for m in self.messages
)
if total_tokens > self.summary_threshold:
# Keep the most recent N messages, summarize the rest
recent_count = min(4, len(self.messages))
old_messages = self.messages[:-recent_count]
recent_messages = self.messages[-recent_count:]
# Build a summary of old messages
old_text = "\n".join([
f"{m['role']}: {m['content'][:200]}" for m in old_messages
])
self.summary += f"\n[Previous context summary]\n{old_text}\n"
self.messages = recent_messages
def get_context_messages(self) -> list:
"""Return messages formatted for the LLM, with summary prepended."""
if self.summary:
return [
{"role": "user", "content": f"Context from earlier in our conversation:\n{self.summary}"},
{"role": "assistant", "content": "Understood. I have this context in mind. Let's continue."},
*self.messages,
]
return self.messages
def get_total_tokens(self) -> int:
base = self.estimate_tokens(self.summary) if self.summary else 0
msgs = sum(self.estimate_tokens(m["content"]) for m in self.messages)
return base + msgs
This ContextManager tracks token usage and automatically compacts older messages into a summary when the threshold is exceeded. The most recent messages are always preserved verbatim, while older context is compressed into a running summary.
Strategy 3: Prioritized Context Ordering
Because of the "lost in the middle" effect, where models pay less attention to information in the center of long contexts, it is important to place the most critical information at the beginning and end of the context. Less critical information should go in the middle.
def build_optimized_context(
system_prompt: str,
user_query: str,
retrieved_docs: list[dict],
tool_results: list[str],
conversation_history: list[dict],
) -> str:
"""
Build a context string optimized for model attention.
Most important content goes at the beginning and end.
"""
# BEGINNING: System instructions and the user's current query
parts = [system_prompt, f"\nCurrent user request: {user_query}\n"]
# MIDDLE: Conversation history and supporting documents
# These are important but less critical than the current task
if conversation_history:
parts.append("--- Conversation History ---")
for msg in conversation_history[-6:]: # Keep last 6 turns
parts.append(f"{msg['role']}: {msg['content']}")
parts.append("")
if retrieved_docs:
parts.append("--- Retrieved Documents ---")
for doc in retrieved_docs:
parts.append(f"[{doc.get('title', 'Untitled')}]: {doc['content']}")
parts.append("")
if tool_results:
parts.append("--- Tool Results ---")
for result in tool_results:
parts.append(result)
parts.append("")
# END: Reiterate the key instruction to anchor the model's focus
parts.append(f"\n--- Reminder ---\nFocus on answering: {user_query}")
parts.append("Use the above context and tool results to provide an accurate, concise response.")
return "\n".join(parts)
By placing the user's query and key instructions at both the start and end of the context, you maximize the model's attention to what matters most.
Strategy 4: Dynamic Tool Selection
When you have many MCP servers connected, listing all available tools can consume significant context. A powerful optimization is to dynamically select which tools to expose based on the current task. You can use a lightweight classifier or keyword matching to filter tools before sending them to the model.
class DynamicToolSelector:
def __init__(self, all_tools: list[dict]):
self.all_tools = all_tools
# Build a simple keyword index for each tool
self.tool_keywords = {}
for tool in all_tools:
keywords = set()
text = f"{tool['name']} {tool['description']}".lower()
for word in text.split():
if len(word) > 3:
keywords.add(word)
self.tool_keywords[tool["name"]] = keywords
def select_relevant_tools(self, user_query: str, max_tools: int = 5) -> list[dict]:
"""Select the most relevant tools based on the user query."""
query_words = set(word.lower() for word in user_query.split() if len(word) > 3)
scored_tools = []
for tool in self.all_tools:
keywords = self.tool_keywords[tool["name"]]
overlap = len(query_words & keywords)
scored_tools.append((overlap, tool))
# Sort by relevance score, take top N
scored_tools.sort(key=lambda x: x[0], reverse=True)
selected = [tool for score, tool in scored_tools[:max_tools] if score > 0]
# Always include a fallback search tool if available
if not any(t["name"] == "search_docs" for t in selected):
search_tool = next((t for t in self.all_tools if t["name"] == "search_docs"), None)
if search_tool:
selected.append(search_tool)
return selected
This approach ensures that the model only sees tools relevant to the current query, dramatically reducing the token overhead of tool definitions in the context window.
Advanced Optimization: Hierarchical Context with MCP
For complex applications, you can implement a hierarchical context strategy where information is organized in tiers. The first tier is always present in the context. The second tier is loaded on demand. The third tier requires explicit model requests through MCP tool calls.
class HierarchicalContextManager:
"""
Three-tier context management:
- Tier 1: Always present (system prompt, core instructions, user query)
- Tier 2: Conditionally loaded (recent conversation, relevant summaries)
- Tier 3: On-demand via MCP (full documents, detailed data, tool results)
"""
def __init__(self, mcp_session):
self.mcp_session = mcp_session
self.tier1 = {} # Always-present context
self.tier2 = {} # Conditionally loaded context
self.tier3_cache = {} # Cached on-demand fetches
def set_tier1(self, key: str, content: str):
"""Set always-present context (system prompt, core instructions)."""
self.tier1[key] = content
def set_tier2(self, key: str, content: str):
"""Set conditionally loaded context (summaries, recent history)."""
self.tier2[key] = content
async def fetch_tier3(self, resource_uri: str) -> str:
"""Fetch on-demand content from MCP server with caching."""
if resource_uri in self.tier3_cache:
return self.tier3_cache[resource_uri]
content = await self.mcp_session.read_resource(resource_uri)
self.tier3_cache[resource_uri] = content
return content
def build_base_context(self, user_query: str, token_budget: int = 4000) -> str:
"""Build the base context within a token budget."""
parts = []
# Always include Tier 1
for key, content in self.tier1.items():
parts.append(f"## {key}\n{content}")
parts.append(f"\n## Current Request\n{user_query}")
# Add Tier 2 content if within budget
current_tokens = sum(len(p) // 4 for p in parts)
for key, content in self.tier2.items():
content_tokens = len(content) // 4
if current_tokens + content_tokens <= token_budget:
parts.append(f"\n## {key}\n{content}")
current_tokens += content_tokens
else:
# Include a truncated version
chars_available = (token_budget - current_tokens) * 4
if chars_available > 100:
parts.append(f"\n## {key} (truncated)\n{content[:chars_available]}...")
break
return "\n".join(parts)
def get_available_resources_hint(self) -> str:
"""Generate a hint about what Tier 3 resources are available."""
return (
"\nAdditional resources are available via MCP. "
"Use the search_docs tool to find relevant documents, "
"then request specific resources by URI to access full content."
)
This hierarchical approach gives you fine-grained control over what enters the context window and when. Tier 1 ensures the model always has essential instructions. Tier 2 provides useful context that fits within a budget. Tier 3 keeps the context lean by deferring detailed data retrieval to explicit model requests.
Best Practices for Context Window Optimization with MCP
1. Keep Tool Descriptions Concise
Every tool you expose consumes context tokens through its name, description, and input schema. Write descriptions that are clear but minimal. Avoid lengthy examples in the description — move those to a separate documentation resource the model can fetch if needed.
# Bad: verbose description wastes tokens
Tool(
name="get_user",
description="This tool retrieves a user from the database by their unique identifier. "
"The user object contains fields like id, name, email, created_at, "
"updated_at, and preferences. You can use this to look up user details "
"when you need to reference a specific user's information. Example: "
"get_user(user_id=12345) returns the full user record.",
inputSchema={...},
)
# Good: concise description saves tokens
Tool(
name="get_user",
description="Retrieve a user record by ID. Returns id, name, email, and preferences.",
inputSchema={...},
)
2. Use Resource URIs as Pointers
Instead of embedding full content in tool results, return resource URIs that the model can fetch later. This is especially useful when a search returns many results — return titles and snippets first, and let the model decide which full documents to read.
3. Implement Caching Aggressively
If the model requests the same resource multiple times within a conversation, cache the result. This avoids redundant MCP calls and keeps the context consistent. The HierarchicalContextManager example above includes a simple cache for Tier 3 resources.
4. Monitor and Measure Token Usage
You cannot optimize what you do not measure. Log the token count of every request, broken down by component (system prompt, tools, history, retrieved content). This data will reveal where your context budget is being spent and where optimizations will have the most impact.
import logging
logger = logging.getLogger("context_monitor")
def log_context_breakdown(system_prompt: str, tools: list, messages: list):
"""Log a detailed breakdown of context token usage."""
system_tokens = len(system_prompt) // 4
tool_tokens = sum(
len(json.dumps(t)) // 4 for t in tools
)
message_tokens = sum(
len(m.get("content", "")) // 4 for m in messages
)
total = system_tokens + tool_tokens + message_tokens
logger.info(f"Context breakdown:")
logger.info(f" System prompt: {system_tokens:>6} tokens")
logger.info(f" Tool definitions: {tool_tokens:>4} tokens")
logger.info(f" Messages: {message_tokens:>6} tokens")
logger.info(f" Total: {total:>6} tokens")
logger.info(f" Tools count: {len(tools)}")
return {
"system": system_tokens,
"tools": tool_tokens,
"messages": message_tokens,
"total": total,
}
5. Design for Graceful Degradation
When the context window is nearly full, your system should degrade gracefully rather than failing. Implement fallback behavior such as summarizing older context, dropping low-priority resources, or splitting a complex query into multiple smaller requests.
6. Separate Static and Dynamic Context
Static context (system instructions, tool schemas) can often be cached at the provider level using prompt caching features. Dynamic context (conversation history, tool results) changes every request. By structuring your prompt so static content comes first, you can take advantage of provider-side caching to reduce both cost and latency.
7. Validate Context Before Sending
Always validate that your context is within the model's limits before sending. Implement a safety check that truncates or compacts context if it exceeds a threshold, rather than letting the API return an error.
def safe_context_for_model(
context: str,
model_limit: int = 128000,
safety_margin: int = 2000,
max_output_tokens: int = 4096,
) -> str:
"""Ensure context fits within model limits with a safety margin."""
available = model_limit - safety_margin - max_output_tokens
context_tokens = len(context) // 4
if context_tokens <= available:
return context
# Truncate from the middle, preserving start and end
chars_available = available * 4
keep_start = chars_available // 3
keep_end = chars_available // 3
truncated = (
context[:keep_start]
+ "\n\n[... context truncated for length ...]\n\n"
+ context[-keep_end:]
)
return truncated
Putting It All Together: A Complete Example
Let's combine all the strategies into a single, cohesive application that uses MCP for context window optimization:
import asyncio
import json
import logging
from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger("mcp-optimized-app")
class OptimizedMCPApplication:
def __init__(self, server_command: list[str], model: str = "claude-sonnet-4-20250514"):
self.server_command = server_command
self.model = model
self.context_manager = ContextManager(max_history_tokens=6000, summary_threshold=4000)
self.tool_selector = None
self.session = None
async def connect(self):
"""Connect to the MCP server and discover capabilities."""
server_params = StdioServerParameters(
command=self.server_command[0],
args=self.server_command[1:],
)
self._client_context = stdio_client(server_params)
read, write = await self._client_context.__aenter__()
self._session_context = ClientSession(read, write)
self.session = await self._session_context.__aenter__()
await self.session.initialize()
# Discover tools
tools_result = await self.session.list_tools()
self.all_tools = [
{
"name": t.name,
"description": t.description,
"input_schema": t.inputSchema,
}
for t in tools_result.tools
]
self.tool_selector = DynamicToolSelector(self.all_tools)
logger.info(f"Discovered {len(self.all_tools)} tools from MCP server")
async def disconnect(self):
"""Clean up MCP connections."""
if self._session_context:
await self._session_context.__aexit__(None, None, None)
if self._client_context:
await self._client_context.__aexit__(None, None, None)
async def chat(self, user_message: str) -> str:
"""Process a user message with optimized context management."""
# Select only relevant tools for this query
relevant_tools = self.tool_selector.select_relevant_tools(user_message, max_tools=5)
logger.info(f"Selected {len(relevant_tools)} relevant tools for query")
# Build the system prompt with compact resource hints
system_prompt = (
"You are a helpful assistant with access to a documentation knowledge base. "
"Use the search_docs tool to find relevant documents before answering. "
"Only read resources that are directly relevant to the user's question."
)
# Get compacted conversation history
history = self.context_manager.get_context_messages()
# Log context breakdown
log_context_breakdown(system_prompt, relevant_tools, history + [{"content": user_message}])
# Make the API call (using Anthropic SDK as example)
import anthropic
client = anthropic.Anthropic()
messages = history + [{"role": "user", "content": user_message}]
response = client.messages.create(
model=self.model,
max_tokens=2048,
system=system_prompt,
tools=relevant_tools,
messages=messages,
)
# Handle tool use loop
while response.stop_reason == "tool_use":
tool_results = []
for block in response.content:
if block.type == "tool_use":
result = await self.session.call_tool(block.name, block.input)
tool_results.append({
"type": "tool_result",
"tool_use_id": block.id,
"content": result.content[0].text if result.content else "",
})
messages.append({"role": "assistant", "content": response.content})
messages.append({"role": "user", "content": tool_results})
response = client.messages.create(
model=self.model,
max_tokens=2048,
system=system_prompt,
tools=relevant_tools,
messages=messages,
)
# Extract final text response
final_text = ""
for block in response.content:
if hasattr(block, "text"):
final_text += block.text
# Update conversation history
self.context_manager.add_message("user", user_message)
self.context_manager.add_message("assistant", final_text)
return final_text
async def main():
app = OptimizedMCPApplication(
server_command=["python", "docs_server.py"],
)
await app.connect()
try:
# Example conversation
questions = [
"How do I handle rate limiting?",
"What authentication method does the API use?",
"Can you remind me what error codes are common?",
]
for question in questions:
print(f"\nUser: {question}")
answer = await app.chat(question)
print(f"Assistant: {answer}")
print(f"Current context tokens: {app.context_manager.get_total_tokens()}")
finally:
await app.disconnect()
if __name__ == "__main__":
asyncio.run(main())
This complete application demonstrates all the optimization strategies working together: lazy resource loading through MCP, dynamic tool selection, context summarization and compaction, and token monitoring. The result is an application that stays within context limits, responds quickly, and costs less to run.
Conclusion
Context window optimization with MCP is a powerful approach to building efficient, scalable AI applications. By leveraging MCP's just-in-time context injection model, you can keep your prompts lean, reduce costs, improve latency, and enhance model performance. The key strategies — lazy resource loading, context summarization, prioritized ordering, dynamic tool selection, and hierarchical context management — work together to ensure that the model always has the right information at the right time without overwhelming its context window. As you build and iterate on your MCP-powered applications, remember to measure token usage continuously, cache aggressively, and design for graceful degradation when limits are approached. With these practices in place, you will be well-equipped to build AI applications that scale gracefully as your data sources, tools, and user base grow.