What Are Multi-Agent Runtimes?
Multi-agent runtimes are orchestration frameworks that coordinate multiple LLM-powered agents to collaborate on complex tasks. Instead of relying on a single monolithic prompt or model call, these systems decompose problems into subtasks handled by specialized agents — each with its own persona, tool access, and reasoning strategy. The runtime manages message routing, state persistence, execution order, and error recovery so agents can work together reliably.
Two of the most prominent open-source multi-agent runtimes today are AutoGen (from Microsoft Research) and LangGraph (from the LangChain ecosystem). Both enable building sophisticated agent systems, but they differ fundamentally in architecture, execution model, and the types of collaboration patterns they express best.
Understanding AutoGen
Core Concepts
AutoGen models multi-agent systems as conversations between agents. Each agent has a role, an LLM backend, and optional tools. Agents communicate by sending and receiving messages through a conversation loop managed by the runtime. The framework supports patterns like two-agent chats, group chats with a moderator, and nested conversations where agents can spawn sub-conversations to solve subproblems.
Key primitives in AutoGen include:
- ConversableAgent — the base class for all agents, handles message generation and reply logic
- AssistantAgent — an agent backed by an LLM, typically used for reasoning and code generation
- UserProxyAgent — represents the human user or a code executor, can trigger tool execution
- GroupChat — orchestrates multiple agents in a shared conversation with a speaker selection mechanism
- GroupChatManager — manages turn-taking and message routing in group chat scenarios
Code Example: AutoGen Two-Agent Conversation
Below is a complete example of setting up a research assistant using AutoGen. The UserProxyAgent acts on behalf of the human, while the AssistantAgent generates code and reasoning to answer the request.
import os
from autogen import AssistantAgent, UserProxyAgent
from autogen.coding import LocalCommandLineCodeExecutor
# Configure the LLM endpoint (using OpenAI-compatible API)
config_list = [
{
"model": "gpt-4o",
"api_key": os.environ.get("OPENAI_API_KEY"),
"base_url": "https://api.openai.com/v1",
}
]
# Create a local code executor that runs Python in a sandboxed workspace
code_executor = LocalCommandLineCodeExecutor(
work_dir="./coding_workspace",
timeout=60,
)
# The assistant agent generates responses and writes code
assistant = AssistantAgent(
name="ResearchAssistant",
system_message="""You are a helpful research assistant.
When asked to analyze data or perform calculations, write Python code to do so.
Explain your reasoning clearly before writing code.
If code execution results are provided, interpret them for the user.""",
llm_config={"config_list": config_list, "temperature": 0.2},
)
# The user proxy represents the human and executes code blocks
user_proxy = UserProxyAgent(
name="User",
human_input_mode="NEVER", # Auto-approve code execution
max_consecutive_auto_reply=5,
code_execution_config={
"executor": code_executor,
"last_n_messages": 3,
},
system_message="You are a user who asks questions and provides feedback.",
)
# Initiate the conversation
user_proxy.initiate_chat(
assistant,
message="""I have a CSV file at ./data/sales.csv with columns:
date, product, quantity, price.
Can you calculate the total revenue per product and identify the top 3 products?""",
)
This example demonstrates AutoGen's conversational model: the user proxy sends the task, the assistant reasons and writes code, the user proxy executes it, and results flow back into the conversation. The framework automatically manages retries if code fails, and the assistant can iteratively refine its solution based on execution feedback.
Code Example: AutoGen Group Chat with Multiple Specialists
For more complex workflows, AutoGen supports group chats where multiple specialist agents collaborate under a moderator:
from autogen import AssistantAgent, UserProxyAgent, GroupChat, GroupChatManager
# Specialist agents
data_analyst = AssistantAgent(
name="DataAnalyst",
system_message="You analyze structured data using Python and pandas.",
llm_config={"config_list": config_list},
)
chart_expert = AssistantAgent(
name="ChartExpert",
system_message="You create matplotlib visualizations based on analysis results.",
llm_config={"config_list": config_list},
)
report_writer = AssistantAgent(
name="ReportWriter",
system_message="You synthesize findings into a polished markdown report.",
llm_config={"config_list": config_list},
)
user_proxy = UserProxyAgent(
name="UserProxy",
human_input_mode="NEVER",
code_execution_config={"executor": code_executor},
)
# Create group chat with all agents
group_chat = GroupChat(
agents=[user_proxy, data_analyst, chart_expert, report_writer],
messages=[],
max_round=12,
speaker_selection_method="auto", # Let the manager choose who speaks next
)
manager = GroupChatManager(
groupchat=group_chat,
llm_config={"config_list": config_list},
)
# Launch the group collaboration
user_proxy.initiate_chat(
manager,
message="""Analyze the sales data in ./data/sales.csv.
Create visualizations of revenue trends and write a summary report.
Save the report as sales_report.md and charts as PNG files.""",
)
In this group chat, the manager dynamically selects which agent speaks next based on conversation context. The DataAnalyst writes analysis code, the ChartExpert creates visualizations from results, and the ReportWriter compiles everything — all within a single managed conversation loop.
Understanding LangGraph
Core Concepts
LangGraph approaches multi-agent coordination from a graph-based perspective. It models agent workflows as directed graphs where nodes represent computation steps (LLM calls, tool executions, or sub-agent invocations) and edges represent control flow — including conditional branching based on state. LangGraph is built on top of LangChain but can be used standalone, and it draws inspiration from state machine and workflow orchestration patterns.
Key primitives in LangGraph include:
- StateGraph — the core abstraction defining a stateful graph with typed state schema
- Nodes — functions or runnables that transform state (agent calls, tools, human inputs)
- Edges — connect nodes; can be normal (always transition) or conditional (branch based on state)
- State — a shared typed dictionary that flows through the graph, accumulating results
- Subgraphs — compose graphs by embedding one graph as a node in another
Code Example: LangGraph Multi-Agent Workflow
The following example builds a multi-agent research workflow where agents are organized as graph nodes with explicit routing logic:
import os
from typing import TypedDict, Annotated, List
from langgraph.graph import StateGraph, END
from langgraph.checkpoint.memory import MemorySaver
from langchain_openai import ChatOpenAI
# Define shared state that flows through the graph
class ResearchState(TypedDict):
task: str
research_notes: Annotated[List[str], "Research findings accumulate here"]
analysis_result: str
final_report: str
next_step: str
# Initialize LLM
llm = ChatOpenAI(model="gpt-4o", temperature=0.2)
# Node 1: Researcher agent gathers information
def researcher_node(state: ResearchState) -> ResearchState:
"""Gathers information about the topic using web search or knowledge."""
prompt = f"""You are a research specialist.
Given this task: {state['task']}
Conduct research and provide 3-5 key findings. Be specific and cite sources where possible."""
response = llm.invoke(prompt)
notes = state.get("research_notes", [])
notes.append(response.content)
return {
**state,
"research_notes": notes,
"next_step": "analyze"
}
# Node 2: Analyst agent processes research
def analyst_node(state: ResearchState) -> ResearchState:
"""Analyzes research findings and draws conclusions."""
notes_text = "\n".join(state.get("research_notes", []))
prompt = f"""You are a data analyst.
Based on these research findings:
{notes_text}
Provide a structured analysis with key insights and patterns identified."""
response = llm.invoke(prompt)
return {
**state,
"analysis_result": response.content,
"next_step": "write_report"
}
# Node 3: Writer agent produces the final output
def writer_node(state: ResearchState) -> ResearchState:
"""Compiles everything into a polished report."""
prompt = f"""You are a professional report writer.
Task: {state['task']}
Research: {state.get('research_notes', [])}
Analysis: {state['analysis_result']}
Write a comprehensive markdown report synthesizing all information.
Include sections: Executive Summary, Findings, Analysis, Recommendations."""
response = llm.invoke(prompt)
return {
**state,
"final_report": response.content,
"next_step": "done"
}
# Routing function: determines which node to visit next
def route_next(state: ResearchState) -> str:
if state.get("next_step") == "analyze":
return "analyst"
elif state.get("next_step") == "write_report":
return "writer"
elif state.get("next_step") == "done":
return END
return END
# Build the graph
builder = StateGraph(ResearchState)
# Add nodes
builder.add_node("researcher", researcher_node)
builder.add_node("analyst", analyst_node)
builder.add_node("writer", writer_node)
# Add edges with conditional routing
builder.add_conditional_edges("researcher", route_next, {
"analyst": "analyst",
"writer": "writer",
END: END,
})
builder.add_conditional_edges("analyst", route_next, {
"writer": "writer",
END: END,
})
builder.add_edge("writer", END)
# Set entry point
builder.set_entry_point("researcher")
# Compile with memory for checkpointing
memory = MemorySaver()
graph = builder.compile(checkpointer=memory)
# Execute the workflow
initial_state = {
"task": "Analyze the impact of remote work on urban real estate markets in 2024",
"research_notes": [],
"analysis_result": "",
"final_report": "",
"next_step": "research"
}
# Run with a thread ID for state persistence
result = graph.invoke(
initial_state,
config={"configurable": {"thread_id": "research_task_1"}}
)
print(result["final_report"])
This LangGraph example shows explicit control flow: the researcher gathers information, the analyst processes it, and the writer produces the final report. Each transition is governed by a routing function that inspects the current state. The graph can be paused, resumed, or forked at any checkpoint thanks to the memory saver.
Code Example: LangGraph with Human-in-the-Loop
LangGraph excels at interruptible workflows. Here's how to add a human approval step before finalizing a report:
from langgraph.graph import StateGraph, END
from langgraph.checkpoint.memory import MemorySaver
# Extended state with approval flag
class ApprovalState(TypedDict):
draft_report: str
approved: bool
final_report: str
def draft_writer(state: ApprovalState) -> ApprovalState:
response = llm.invoke("Write a draft report on the given topic.")
return {**state, "draft_report": response.content}
def human_review(state: ApprovalState) -> ApprovalState:
"""This node interrupts execution and waits for human input."""
# The graph will pause here — in production, this triggers a UI prompt
# For this example, we simulate approval via state
return state # State includes 'approved' set externally
def finalize(state: ApprovalState) -> ApprovalState:
if state.get("approved"):
return {**state, "final_report": state["draft_report"] + "\n\n[APPROVED]"}
return {**state, "final_report": "Draft rejected — needs revision"}
builder = StateGraph(ApprovalState)
builder.add_node("draft", draft_writer)
builder.add_node("human_review", human_review)
builder.add_node("finalize", finalize)
builder.add_edge("draft", "human_review")
builder.add_conditional_edges(
"human_review",
lambda s: "finalize" if s.get("approved") else END,
{"finalize": "finalize", END: END}
)
builder.add_edge("finalize", END)
builder.set_entry_point("draft")
memory = MemorySaver()
graph = builder.compile(
checkpointer=memory,
interrupt_before=["human_review"] # Pause before this node
)
# First invocation — runs until human_review node
result = graph.invoke(
{"draft_report": "", "approved": False, "final_report": ""},
config={"configurable": {"thread_id": "report_1"}}
)
# Graph pauses here. In production, send draft to human for review.
# Human approves and resumes with updated state
result = graph.invoke(
None, # State is loaded from checkpoint
config={"configurable": {"thread_id": "report_1"}},
state={"approved": True} # Override specific fields
)
print(result["final_report"])
This interrupt pattern is native to LangGraph's architecture. You can pause execution before any node, serialize the state, present it to a human reviewer, and resume with modifications — all while maintaining a persistent execution history.
Why This Choice Matters
Choosing between AutoGen and LangGraph shapes your system's reliability, debuggability, and extensibility in fundamental ways:
- Execution transparency — LangGraph's explicit graph structure makes it straightforward to visualize, trace, and audit exactly what path execution took. AutoGen's conversation-based model is more emergent; the flow depends on agent responses and moderator decisions that can be harder to predict.
- Error recovery — LangGraph's checkpointing allows resuming from any node after a failure, with full state restoration. AutoGen handles errors through conversation retry mechanisms, which works well for code execution errors but offers less granular control for structural failures.
- Parallelism and branching — LangGraph natively supports fan-out patterns where multiple nodes execute concurrently and merge results. AutoGen's group chat is inherently sequential (one speaker at a time), though nested conversations can approximate parallelism.
- Integration surface — AutoGen integrates deeply with code execution environments and offers built-in Docker sandboxing. LangGraph integrates seamlessly with the broader LangChain ecosystem including vector stores, retrievers, and tool libraries.
- Learning curve — AutoGen's conversational metaphor is intuitive and quick to prototype. LangGraph requires thinking in terms of state machines and graph topology, which offers more control but demands more upfront design.
How to Choose: Decision Framework
Use this structured framework to evaluate which runtime fits your project:
Choose AutoGen when:
- Your workflow is primarily conversational or collaborative — agents debating, refining, and iterating on each other's outputs
- You need code generation and execution as a core capability, especially with sandboxed environments
- You want rapid prototyping with minimal structural boilerplate
- Your agents need to autonomously decide who speaks next in a multi-participant dialogue
- You're building applications like automated data analysis, code review assistants, or collaborative problem-solving systems
Choose LangGraph when:
- Your workflow follows a defined process or pipeline with clear stages and decision points
- You need human-in-the-loop interrupts at specific steps with approval gates
- Deterministic control flow matters — you want explicit routing rules rather than emergent agent behavior
- You require persistence and resumability across long-running or fault-prone operations
- You're building applications like document processing pipelines, approval workflows, or staged reasoning systems
Hybrid approach
Many production systems combine both. Use LangGraph for the outer orchestration layer (routing, state management, human approvals) and embed AutoGen group chats as subgraph nodes for the inner collaborative reasoning phases where emergent agent interaction adds value.
Best Practices
For AutoGen
- Set clear agent personas — each agent's system message should precisely define its role, boundaries, and expected output format. Vague personas lead to rambling conversations
- Cap conversation rounds — use
max_roundin group chats andmax_consecutive_auto_replyin two-agent chats to prevent infinite loops - Use speaker selection methods wisely —
"auto"mode lets the manager LLM choose speakers (flexible but costly), while"round_robin"gives predictable order (rigid but reliable) - Leverage nested conversations — when an agent needs to solve a subproblem privately (without polluting the main chat), spawn a nested conversation with a temporary assistant
- Monitor token usage — group chats accumulate full conversation history, which grows quickly. Implement message filtering or summarization for long-running sessions
For LangGraph
- Design state schema carefully — the shared state TypedDict is your contract between nodes. Include all fields needed for routing decisions and use
Annotatedtypes for fields that accumulate (like lists) - Keep nodes pure and focused — each node should do one thing well. Avoid monolithic nodes that mix planning, execution, and formatting
- Use checkpointers strategically —
MemorySaverfor development, but switch toSqliteSaveror a custom persistent backend for production - Implement comprehensive routing logic — handle edge cases in conditional edge functions: missing data, error states, and early termination conditions
- Test with small graphs first — build and debug a minimal 2-node graph before expanding to complex topologies. Graph structure bugs are harder to diagnose than flat conversation issues
- Leverage subgraphs for modularity — encapsulate reusable agent patterns (like a "research → analyze → write" sequence) as subgraphs that can be composed into larger workflows
General multi-agent best practices
- Log everything — multi-agent systems produce complex interaction traces. Implement structured logging with agent names, timestamps, and state snapshots for debugging
- Implement circuit breakers — if an agent retries the same action more than N times without progress, escalate or route to a human
- Validate agent outputs — never trust raw LLM output from one agent as input to another without at least structural validation (type checks, required fields, format compliance)
- Start simple, then distribute — begin with a single capable agent handling all tasks. Only split into multiple agents when you observe concrete bottlenecks that specialization resolves
Conclusion
AutoGen and LangGraph represent two distinct philosophies for multi-agent orchestration. AutoGen embraces the conversational metaphor — agents talk to each other, and coordination emerges from dialogue. LangGraph embraces explicit state machines — you design the flow, and agents execute within that structure. Neither is universally superior; the right choice depends on whether your problem is best modeled as a conversation or as a pipeline.
For exploratory data analysis, collaborative reasoning, and scenarios where agent autonomy and emergent behavior are assets, AutoGen's conversational model shines. For production workflows requiring audit trails, human approvals, deterministic routing, and fault recovery, LangGraph's graph-based architecture provides the necessary control. Understanding both frameworks — and recognizing when to reach for each — is becoming an essential skill for developers building reliable AI agent systems.