Context Window Optimization with OpenAI Agents SDK: Complete Guide

Context Window Optimization with OpenAI Agents SDK: Complete Guide

As AI agents take on increasingly complex tasks, the context window — the limited memory an LLM can reference at any given moment — becomes one of the most critical bottlenecks in production systems. The OpenAI Agents SDK provides powerful primitives for building autonomous agents, but without deliberate context management, your agents will burn through tokens, slow down responses, and eventually degrade in reasoning quality. This guide walks through everything you need to know to keep your agents sharp, fast, and cost-effective.

What Is Context Window Optimization?

Context window optimization is the practice of managing what information enters an LLM's prompt at each step of an agent's lifecycle. Every model has a fixed maximum context size — for example, GPT-4o supports 128,000 tokens — but filling that window indiscriminately causes several problems:

• Higher costs: Token-based pricing means every token in the context is billed on each request.
• Latency: Larger prompts take longer to process and generate.
• Lost in the middle: Models often perform worse on information buried in the middle of long contexts.
• Reasoning degradation: Excessive noise makes it harder for the model to focus on what matters.

Optimization means deliberately curating, compressing, and structuring context so the agent sees exactly what it needs — no more, no less.

Why It Matters for the OpenAI Agents SDK

The OpenAI Agents SDK orchestrates multi-step agent workflows where each step typically appends new information: tool outputs, intermediate reasoning, sub-agent results, and user messages. Without intervention, the conversation history grows linearly, and multi-agent handoffs can multiply that growth. In long-running sessions — customer support bots, research agents, coding assistants — this becomes unsustainable within minutes.

The SDK exposes several hooks that make optimization tractable: lifecycle hooks, handoff controls, guardrails, and the ability to inject dynamic context. Combined with external techniques like retrieval, summarization, and structured memory, you can build agents that stay responsive even after hours of operation.

Understanding the Context Lifecycle in the Agents SDK

How Context Accumulates

Every time an agent runs, the SDK assembles a prompt from several sources:

• The system prompt (agent instructions)
• The full conversation history (messages exchanged so far)
• Tool definitions and their schemas
• Tool call results appended to the history
• Handoff metadata when control transfers between agents

Each loop iteration of the agent's reasoning cycle adds tokens. A 10-step tool-use sequence can easily balloon a 500-token starting prompt into 20,000 tokens of context.

Measuring Context Usage

Before optimizing, you need visibility. The SDK's RunResult object exposes usage statistics you can inspect after each run.

import asyncio
from agents import Agent, Runner

agent = Agent(
    name="ResearchAgent",
    instructions="You are a research assistant. Answer concisely.",
)

async def main():
    result = await Runner.run(agent, "Summarize the causes of the French Revolution.")
    print("Output:", result.final_output)
    print("Input tokens:", result.context_wrapper.usage.input_tokens)
    print("Output tokens:", result.context_wrapper.usage.output_tokens)
    print("Total tokens:", result.context_wrapper.usage.total_tokens)

asyncio.run(main())

Logging these metrics across runs gives you a baseline. From there, you can identify which steps contribute the most context bloat.

Strategies for Context Window Optimization

1. Write Lean System Prompts

The system prompt is sent with every single request. Bloated instructions — long examples, redundant rules, verbose formatting guides — multiply cost across every turn. Keep instructions tight and behavioral rather than encyclopedic.

# Bad: verbose, example-heavy
instructions = """
You are a helpful customer support agent for Acme Corp.
Acme Corp sells widgets, gadgets, and sprockets.
Widgets come in three sizes: small, medium, large.
Gadgets have a 30-day return policy.
Sprockets are non-refundable but come with a 1-year warranty.
When a customer asks about returns, explain the policy in detail.
When a customer asks about warranties, explain the warranty terms.
Always greet the customer politely.
Always thank them at the end.
... (500 more tokens of static info)
"""

# Good: lean, behavior-focused, offload facts to tools
instructions = """
You are Acme Corp's support agent. Greet briefly, then help.
Use the `lookup_product` tool for product details,
`lookup_policy` tool for return/warranty info, and
`escalate` tool when unsure. Keep replies under 80 words.
"""

Move static reference data into tools the agent can call on demand. This keeps the always-on context small while preserving access to full information.

2. Summarize Conversation History

For long-running conversations, periodically replace older messages with a compressed summary. The Agents SDK lets you implement this through lifecycle hooks or by manually managing the message list between runs.

from agents import Agent, Runner
from agents.memory import Session
from openai import AsyncOpenAI
import asyncio

client = AsyncOpenAI()

async def summarize_messages(messages, max_messages_to_keep=6):
    """Keep the most recent messages; summarize the rest."""
    if len(messages) <= max_messages_to_keep:
        return messages

    to_summarize = messages[:-max_messages_to_keep]
    recent = messages[-max_messages_to_keep:]

    transcript = "\n".join(
        f"{m['role']}: {m['content']}" for m in to_summarize
        if isinstance(m.get("content"), str)
    )

    summary_resp = await client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": "Summarize this conversation in under 200 tokens. Preserve key facts, decisions, and any unresolved questions."},
            {"role": "user", "content": transcript},
        ],
    )
    summary = summary_resp.choices[0].message.content

    return [
        {"role": "system", "content": f"Earlier conversation summary: {summary}"}
    ] + recent

agent = Agent(name="ChatAgent", instructions="You are a helpful assistant.")

async def chat_loop():
    messages = []
    while True:
        user_input = input("You: ")
        if user_input.lower() in {"exit", "quit"}:
            break
        messages.append({"role": "user", "content": user_input})

        result = await Runner.run(agent, input=messages)
        messages = result.to_input_list()
        print("Agent:", result.final_output)

        # Optimize context after each turn
        messages = await summarize_messages(messages)

asyncio.run(chat_loop())

This approach caps context growth: no matter how long the conversation runs, the history stays bounded by the summary size plus the recent window.

3. Use Retrieval Instead of Stuffing

When an agent needs access to large knowledge bases, never paste the documents into the prompt. Instead, expose a retrieval tool that fetches only relevant chunks.

from agents import Agent, function_tool
from typing import List

# Pretend this is backed by a vector database
DOCUMENT_STORE = {
    "doc_1": "Acme's return policy allows returns within 30 days for gadgets...",
    "doc_2": "Sprockets carry a 1-year manufacturer warranty...",
    "doc_3": "Widget sizes: small (5cm), medium (10cm), large (15cm)...",
}

@function_tool
def search_knowledge_base(query: str) -> List[str]:
    """Search the product knowledge base for relevant documents."""
    # In production: embed query, do similarity search, return top-k chunks
    results = []
    query_lower = query.lower()
    for doc_id, content in DOCUMENT_STORE.items():
        if any(word in content.lower() for word in query_lower.split()):
            results.append(content)
    return results[:3]

support_agent = Agent(
    name="SupportAgent",
    instructions=(
        "You are Acme's support agent. Use `search_knowledge_base` "
        "to find product info before answering. Be concise."
    ),
    tools=[search_knowledge_base],
)

The agent only pulls in documents when needed, and only the most relevant slices — keeping context focused on the current question.

4. Trim Tool Output Aggressively

Tool outputs are a common source of context bloat. A database query returning 50 rows, or a web scraper returning a full page, can dump thousands of tokens into history. Always shape tool outputs before returning them.

import json
from agents import function_tool

@function_tool
def query_orders(user_id: str) -> str:
    """Fetch recent orders for a user. Returns a compact summary."""
    # Simulated raw query result
    raw_orders = [
        {"id": "ORD-001", "product": "Widget", "price": 19.99, "status": "shipped", "date": "2024-11-01", "tracking": "1Z999AA10123456784", "warehouse": "NJ-3", "carrier": "UPS"},
        {"id": "ORD-002", "product": "Gadget", "price": 49.99, "status": "delivered", "date": "2024-10-15", "tracking": "1Z999AA10123456790", "warehouse": "CA-1", "carrier": "UPS"},
        {"id": "ORD-003", "product": "Sprocket", "price": 12.50, "status": "processing", "date": "2024-11-20", "tracking": None, "warehouse": "TX-2", "carrier": None},
    ]

    # Compact: only fields the agent likely needs
    compact = [
        {"id": o["id"], "product": o["product"], "status": o["status"], "date": o["date"]}
        for o in raw_orders
    ]
    return json.dumps(compact)

Notice we dropped tracking numbers, warehouse codes, and carrier info — details the agent can fetch separately if a customer specifically asks. This single pattern can reduce tool output size by 60–80%.

5. Use Sub-Agents to Isolate Context

The Agents SDK supports handoffs, where one agent delegates to another. Each sub-agent runs with its own context window, so you can isolate heavy operations — like analyzing a long document — inside a sub-agent whose bloated context never pollutes the main agent's history.

from agents import Agent, Runner
import asyncio

# Sub-agent: does the heavy lifting, produces a compact result
document_analyzer = Agent(
    name="DocumentAnalyzer",
    instructions=(
        "You analyze documents and return a structured summary. "
        "Always respond with a JSON object containing 'summary' "
        "(string, max 100 words) and 'key_points' (array of strings, max 5 items)."
    ),
)

# Main agent: stays lean, delegates heavy work
research_agent = Agent(
    name="ResearchAgent",
    instructions=(
        "You help users research topics. For any document analysis, "
        "hand off to the DocumentAnalyzer. Use only its final output."
    ),
    handoffs=[document_analyzer],
)

async def main():
    long_document = "..."  # imagine 10,000 tokens of text
    user_msg = f"Please analyze this document and tell me the key takeaways:\n\n{long_document}"

    result = await Runner.run(research_agent, user_msg)
    print(result.final_output)

asyncio.run(main())

After the handoff completes, the main agent only sees the sub-agent's compact final output — not the full document or intermediate reasoning. This is one of the most powerful context management patterns in the SDK.

6. Implement Sliding Window Memory with Sessions

For stateful agents that persist across multiple user interactions, use the SDK's session abstraction combined with a sliding window strategy.

from agents import Agent, Runner
from agents.memory import Session
import asyncio

class SlidingWindowSession(Session):
    def __init__(self, max_messages=20):
        self._messages = []
        self._max = max_messages

    async def get_items(self, limit=None):
        return self._messages[-(limit or self._max):]

    async def add_items(self, items):
        self._messages.extend(items)
        # Enforce sliding window
        if len(self._messages) > self._max:
            self._messages = self._messages[-self._max:]

    async def pop_item(self):
        if self._messages:
            return self._messages.pop(0)
        return None

    async def clear(self):
        self._messages = []

agent = Agent(
    name="PersistentAgent",
    instructions="You are a helpful assistant that remembers recent context.",
)

async def main():
    session = SlidingWindowSession(max_messages=10)

    # Multiple turns — only the last 10 messages are retained
    for i in range(15):
        result = await Runner.run(
            agent,
            input=f"Turn {i}: Tell me something interesting about number {i}.",
            session=session,
        )
        print(f"Turn {i}:", result.final_output[:80])

asyncio.run(main())

This guarantees a hard ceiling on context size regardless of how many turns occur.

Best Practices

• Measure first: Always log token usage before optimizing. You cannot improve what you don't track.
• Prefer tools over prompt-stuffing: Any reference data that isn't needed on every turn belongs in a tool, not the system prompt.
• Summarize proactively, not reactively: Don't wait until you hit the context limit. Summarize when history exceeds a comfortable threshold (e.g., 50% of your target window).
• Use cheaper models for summarization: Route compression tasks to gpt-4o-mini or similar — summarization doesn't require frontier reasoning.
• Structure tool outputs: Return JSON, not prose. Structured data is more token-efficient and easier for the model to parse.
• Isolate heavy contexts in sub-agents: Use handoffs to quarantine operations that inherently require large inputs.
• Cache tool results: If the same tool is called with the same arguments, return cached results instead of re-fetching and re-injecting.
• Set explicit output limits: Instruct agents to keep responses concise, and enforce with guardrails where possible.
• Test with realistic session lengths: A 2-turn test won't reveal context issues. Always test with sessions that mirror real usage duration.

Putting It All Together

Here's a combined example that applies several optimization techniques simultaneously:

import json
import asyncio
from agents import Agent, Runner, function_tool
from openai import AsyncOpenAI

client = AsyncOpenAI()

@function_tool
def search_docs(query: str) -> str:
    """Search internal docs. Returns compact JSON results."""
    # Vector search in production
    fake_results = [
        {"title": "Return Policy", "snippet": "30-day returns for gadgets."},
        {"title": "Warranty Terms", "snippet": "1-year warranty on sprockets."},
    ]
    return json.dumps(fake_results)

async def compress_history(messages, keep_recent=4):
    if len(messages) <= keep_recent:
        return messages
    old = messages[:-keep_recent]
    recent = messages[-keep_recent:]
    transcript = "\n".join(
        m.get("content", "") for m in old if isinstance(m.get("content"), str)
    )
    resp = await client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": "Summarize in 150 tokens. Keep facts and decisions."},
            {"role": "user", "content": transcript},
        ],
    )
    summary = resp.choices[0].message.content
    return [{"role": "system", "content": f"Prior context: {summary}"}] + recent

agent = Agent(
    name="OptimizedSupportAgent",
    instructions=(
        "Acme support agent. Greet briefly. Use `search_docs` for product info. "
        "Keep replies under 80 words. Escalate if unsure."
    ),
    tools=[search_docs],
)

async def run_session():
    messages = []
    test_inputs = [
        "What's your return policy for gadgets?",
        "How long is the warranty on sprockets?",
        "Can I return a sprocket I bought last month?",
        "What about a gadget I bought two months ago?",
        "Summarize everything we've discussed.",
    ]
    for user_input in test_inputs:
        messages.append({"role": "user", "content": user_input})
        result = await Runner.run(agent, input=messages)
        messages = result.to_input_list()
        print(f"User: {user_input}")
        print(f"Agent: {result.final_output}\n")
        messages = await compress_history(messages)

asyncio.run(run_session())

This example combines lean instructions, tool-based retrieval, history compression, and concise output directives — the four pillars of a well-optimized agent.

Conclusion

Context window optimization is not a one-time configuration but an ongoing discipline that scales with your agent's complexity. By writing lean system prompts, offloading reference data to tools, compressing conversation history, trimming tool outputs, isolating heavy work in sub-agents, and enforcing sliding-window memory, you can build agents on the OpenAI Agents SDK that remain fast, affordable, and accurate even in long-running sessions. Start by measuring your current token usage, identify the biggest sources of bloat, and apply the techniques in this guide incrementally — your users, your budget, and your agents' reasoning quality will all benefit.