Building a Code Review Agent with AutoGen: Complete Guide

What is a Code Review Agent?

A Code Review Agent is an AI-powered assistant that automatically analyzes source code changes, identifies potential issues, and provides actionable feedback — much like a human reviewer would during a pull request review cycle. Built with AutoGen, Microsoft's multi-agent conversation framework, this agent can collaborate with other specialized agents (such as a code writer agent or a testing agent) to form a complete automated development pipeline.

Unlike static linters or simple rule-based analyzers, a code review agent built with AutoGen can engage in conversational, multi-turn reasoning. It doesn't just flag syntax errors — it can question architectural decisions, suggest alternative implementations, verify adherence to coding standards, detect security vulnerabilities, and even negotiate improvements with a code-generating agent in real time.

At its core, the agent uses a Large Language Model (LLM) configured through AutoGen's agent abstraction layer. You define the agent's persona, its review criteria, and the conversation patterns it should follow. The result is a flexible, extensible reviewer that can be tailored to your team's specific code review checklist, language ecosystem, and quality bar.

Why AutoGen for Code Review Matters

Manual code review is time-consuming, inconsistent, and often becomes a bottleneck in fast-moving development teams. Studies show that human reviewers miss up to 40% of defects in large diffs due to fatigue or cognitive overload. AutoGen-based review agents address this by offering:

• Consistency: The agent applies the same rigorous standards to every review, eliminating variability between human reviewers.
• Speed: Reviews complete in seconds, not hours or days, unblocking developers immediately after a commit or push.
• Scalability: A single agent can review hundreds of pull requests simultaneously across multiple repositories.
• Multi-Agent Collaboration: With AutoGen, the review agent doesn't work in isolation — it can loop in a security specialist agent, a performance analyst agent, or a test generation agent as needed.
• Auditability: Every review conversation is logged, creating a traceable record of what was flagged, why, and how it was resolved.

AutoGen specifically shines because it abstracts away the complexity of managing LLM conversations. You don't need to hand-craft complex prompt chains or manage state between calls — AutoGen handles turn-taking, context management, and tool execution through its conversation-driven programming model. This lets you focus on what the agent should review rather than how to orchestrate the underlying LLM calls.

Setting Up Your Environment

Before building the agent, install the required dependencies. You'll need Python 3.10+, the AutoGen package, and an LLM provider (we'll use OpenAI's GPT-4o in this tutorial, but AutoGen supports many backends including Azure OpenAI, Anthropic, and local models via LiteLLM).

# Create and activate a virtual environment
python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install AutoGen and dependencies
pip install pyautogen
pip install openai
pip install python-dotenv

Create a .env file to store your API key securely:

# .env
OPENAI_API_KEY=sk-your-key-here
OPENAI_MODEL=gpt-4o

Now verify your setup with a minimal AutoGen script that creates two agents and runs a simple conversation:

import os
import autogen
from dotenv import load_dotenv

load_dotenv()

config_list = [
    {
        "model": os.getenv("OPENAI_MODEL", "gpt-4o"),
        "api_key": os.getenv("OPENAI_API_KEY"),
    }
]

# Create a simple assistant and user proxy
assistant = autogen.AssistantAgent(
    name="assistant",
    llm_config={"config_list": config_list},
)

user_proxy = autogen.UserProxyAgent(
    name="user",
    human_input_mode="NEVER",
    code_execution_config=False,
)

# Run a quick test
user_proxy.initiate_chat(
    assistant,
    message="Say 'Code review agent setup verified!' and nothing else.",
    max_turns=1,
)
print("Setup successful!")

Building the Code Review Agent Step by Step

1. Defining the Review Agent's Persona

The most critical step is crafting the system prompt that defines how the agent thinks. A well-designed persona includes the agent's role, its review philosophy, the specific criteria it checks, and the output format it should follow. Here's a production-ready system prompt for a Python-focused code reviewer:

CODE_REVIEWER_SYSTEM_PROMPT = """You are a senior software engineer performing a thorough code review.
Your name is CodeReviewBot. You review Python code changes with the following principles:

## Review Philosophy
- Be constructive, not judgmental. Frame feedback as suggestions, not commands.
- Prioritize issues by severity: CRITICAL (security, data loss), HIGH (bugs, incorrect logic),
  MEDIUM (performance, maintainability), LOW (style, naming).
- Always explain WHY a change is recommended, not just WHAT to change.
- If the code is excellent, say so clearly and concisely.

## Review Checklist
1. **Security**: Check for SQL injection, hardcoded secrets, unsafe deserialization,
   path traversal, missing input validation, insecure cryptography.
2. **Correctness**: Off-by-one errors, edge cases, null/None handling, type mismatches,
   race conditions, improper error handling.
3. **Performance**: N+1 queries, unnecessary allocations, blocking I/O in async contexts,
   missing caching opportunities, inefficient data structures.
4. **Maintainability**: Code duplication, overly complex functions, missing docstrings,
   magic numbers, tight coupling, violation of SOLID principles.
5. **Testing**: Missing test coverage for edge cases, untestable code patterns,
   improper mocking, brittle assertions.

## Output Format
For each issue found, use this structure:
[SEVERITY]: Brief title
File: 
Line: 
Issue: 
Suggestion: 
If no issues are found, respond with:
[PASS]: No issues found. Code looks good!
Be thorough but concise. Review every line of the diff provided."""

2. Creating the Review Agent with AutoGen

With the persona defined, instantiate the review agent. We'll use AssistantAgent — it's designed for LLM-powered roles that don't need to execute code or interact with the human user directly. The agent carries the system prompt as part of its configuration and will maintain conversation context across multiple turns.

import autogen

def create_code_reviewer(config_list):
    """Create and return a configured code review agent."""
    
    reviewer = autogen.AssistantAgent(
        name="CodeReviewBot",
        system_message=CODE_REVIEWER_SYSTEM_PROMPT,
        llm_config={
            "config_list": config_list,
            "temperature": 0.1,  # Low temperature for consistent, analytical output
            "timeout": 120,
            "max_tokens": 4096,
        },
        # The agent can request human input if it encounters ambiguous code
        human_input_mode="NEVER",
    )
    
    return reviewer

3. Building the Code Fetching Tool

For the agent to review actual code, it needs access to files. AutoGen supports tool use — you can register Python functions that the LLM can call during the conversation. Let's create a tool that reads file contents and another that fetches a simulated "diff" (the changes to review).

def read_file(file_path: str) -> str:
    """Read and return the contents of a file at the given path."""
    try:
        with open(file_path, "r", encoding="utf-8") as f:
            content = f.read()
        return f"File: {file_path}\nLines: {content.count(chr(10)) + 1}\n\npython\n{content}\n"
    except FileNotFoundError:
        return f"Error: File '{file_path}' not found."
    except Exception as e:
        return f"Error reading file: {str(e)}"

def read_diff(diff_description: str) -> str:
    """Simulate fetching a diff. In production, this would call the GitHub/GitLab API
    or run 'git diff' to get the actual changes."""
    # For the tutorial, we'll load from a local diff file if it exists
    try:
        with open("review_diff.txt", "r", encoding="utf-8") as f:
            return f.read()
    except FileNotFoundError:
        return f"No diff file found. Please provide the code to review directly."

4. Registering Tools with the Agent

AutoGen allows you to register these functions as tools that the LLM can invoke. When the agent decides it needs file contents, it will emit a tool call, the framework executes the function, and the result is fed back into the conversation — all seamlessly.

from autogen import register_function

def configure_reviewer_tools(reviewer_agent, executor_agent):
    """Register file-reading tools for the reviewer to use."""
    
    register_function(
        read_file,
        caller=reviewer_agent,
        executor=executor_agent,
        name="read_file",
        description="Read the contents of a source code file. Provide the full file path.",
    )
    
    register_function(
        read_diff,
        caller=reviewer_agent,
        executor=executor_agent,
        name="read_diff",
        description="Fetch the current diff/patch to review. Call this when you need to see what code changed.",
    )
    
    return reviewer_agent

5. Creating the User Proxy (Orchestrator)

The UserProxyAgent acts as the human's representative in the conversation. It can execute code (if you enable that), relay messages, and most importantly for our use case — it executes the tool functions on behalf of the LLM. We'll configure it with code execution disabled since the review agent only needs to read files, not run them.

def create_user_proxy(config_list):
    """Create a user proxy agent that executes tools for the reviewer."""
    
    user_proxy = autogen.UserProxyAgent(
        name="UserProxy",
        human_input_mode="NEVER",  # Fully automated; set to "TERMINATE" for interactive mode
        max_consecutive_auto_reply=10,
        code_execution_config=False,  # We don't need code execution for review
        llm_config=False,  # No LLM needed for the proxy — it just relays and executes tools
        # If the reviewer signals completion, the proxy can terminate
        is_termination_msg=lambda msg: "REVIEW_COMPLETE" in msg.get("content", ""),
    )
    
    return user_proxy

6. Assembling the Complete Code Review System

Now we tie everything together. The full system creates both agents, registers the tools, and initiates a review conversation. The key insight is that the conversation itself is the review process — the review agent thinks step by step, calls tools to fetch code, analyzes what it sees, and produces structured feedback.

import os
import autogen
from dotenv import load_dotenv
from autogen import AssistantAgent, UserProxyAgent, register_function

load_dotenv()

# ========== Configuration ==========
config_list = [
    {
        "model": os.getenv("OPENAI_MODEL", "gpt-4o"),
        "api_key": os.getenv("OPENAI_API_KEY"),
    }
]

# ========== System Prompt ==========
CODE_REVIEWER_SYSTEM_PROMPT = """You are a senior software engineer performing a thorough code review.
Your name is CodeReviewBot. You review Python code changes with the following principles:

## Review Philosophy
- Be constructive, not judgmental. Frame feedback as suggestions, not commands.
- Prioritize issues by severity: CRITICAL (security, data loss), HIGH (bugs, incorrect logic),
  MEDIUM (performance, maintainability), LOW (style, naming).
- Always explain WHY a change is recommended, not just WHAT to change.
- If the code is excellent, say so clearly and concisely.

## Review Checklist
1. **Security**: Check for SQL injection, hardcoded secrets, unsafe deserialization,
   path traversal, missing input validation, insecure cryptography.
2. **Correctness**: Off-by-one errors, edge cases, null/None handling, type mismatches,
   race conditions, improper error handling.
3. **Performance**: N+1 queries, unnecessary allocations, blocking I/O in async contexts,
   missing caching opportunities, inefficient data structures.
4. **Maintainability**: Code duplication, overly complex functions, missing docstrings,
   magic numbers, tight coupling, violation of SOLID principles.
5. **Testing**: Missing test coverage for edge cases, untestable code patterns,
   improper mocking, brittle assertions.

## Output Format
For each issue found, use this structure:
[SEVERITY]: Brief title
File: 
Line: 
Issue: 
Suggestion: 
If no issues are found, respond with:
[PASS]: No issues found. Code looks good!
After completing your review, end your message with: REVIEW_COMPLETE

Be thorough but concise. Review every line of the diff provided.""" 

# ========== Tool Functions ==========
def read_file(file_path: str) -> str:
    """Read and return the contents of a file at the given path."""
    try:
        with open(file_path, "r", encoding="utf-8") as f:
            content = f.read()
        return f"File: {file_path}\nLines: {content.count(chr(10)) + 1}\n\npython\n{content}\n"
    except FileNotFoundError:
        return f"Error: File '{file_path}' not found."
    except Exception as e:
        return f"Error reading file: {str(e)}"

def read_diff(diff_description: str) -> str:
    """Fetch the current diff/patch to review."""
    try:
        with open("review_diff.txt", "r", encoding="utf-8") as f:
            return f.read()
    except FileNotFoundError:
        return "No diff file found. Please provide the code to review directly."

# ========== Agent Creation ==========
reviewer = AssistantAgent(
    name="CodeReviewBot",
    system_message=CODE_REVIEWER_SYSTEM_PROMPT,
    llm_config={
        "config_list": config_list,
        "temperature": 0.1,
        "timeout": 120,
        "max_tokens": 4096,
    },
    human_input_mode="NEVER",
)

user_proxy = UserProxyAgent(
    name="UserProxy",
    human_input_mode="NEVER",
    max_consecutive_auto_reply=10,
    code_execution_config=False,
    llm_config=False,
    is_termination_msg=lambda msg: "REVIEW_COMPLETE" in msg.get("content", ""),
)

# ========== Register Tools ==========
register_function(
    read_file,
    caller=reviewer,
    executor=user_proxy,
    name="read_file",
    description="Read the contents of a source code file. Provide the full file path.",
)

register_function(
    read_diff,
    caller=reviewer,
    executor=user_proxy,
    name="read_diff",
    description="Fetch the current diff/patch to review. Call when you need to see changed code.",
)

# ========== Initiate Review ==========
def start_review(instructions: str = None):
    """Start the code review process."""
    if instructions is None:
        instructions = (
            "Please review the code changes in the current diff. "
            "Use read_diff to fetch the changes, then read_file to examine "
            "any files you need full context for. Provide a thorough review "
            "following your checklist and output format."
        )
    
    user_proxy.initiate_chat(
        reviewer,
        message=instructions,
        max_turns=10,
    )

if __name__ == "__main__":
    start_review()

7. Testing the Agent with Sample Code

Create a sample diff file to test the agent. This simulates what you'd get from a real pull request:

# review_diff.txt
diff --git a/src/api/handlers.py b/src/api/handlers.py
index abc123..def456 100644
--- a/src/api/handlers.py
+++ b/src/api/handlers.py
@@ -15,6 +15,12 @@ def get_user(user_id: int) -> dict:
     """Fetch user by ID from database."""
     query = f"SELECT * FROM users WHERE id = {user_id}"
     cursor.execute(query)
     return cursor.fetchone()
+
+def delete_user(user_id: str) -> bool:
+    """Delete a user account."""
+    query = f"DELETE FROM users WHERE id = '{user_id}'"
+    cursor.execute(query)
+    conn.commit()
+    return True

Now create the actual source file that the agent will read for full context:

# src/api/handlers.py (sample file)
import sqlite3

conn = sqlite3.connect("app.db")
cursor = conn.cursor()

def get_all_users() -> list:
    """Return all users."""
    cursor.execute("SELECT * FROM users")
    return cursor.fetchall()

def get_user(user_id: int) -> dict:
    """Fetch user by ID from database."""
    query = f"SELECT * FROM users WHERE id = {user_id}"
    cursor.execute(query)
    return cursor.fetchone()

def delete_user(user_id: str) -> bool:
    """Delete a user account."""
    query = f"DELETE FROM users WHERE id = '{user_id}'"
    cursor.execute(query)
    conn.commit()
    return True

def update_email(user_id: int, new_email: str) -> bool:
    """Update user email."""
    cursor.execute(
        "UPDATE users SET email = ? WHERE id = ?",
        (new_email, user_id)
    )
    conn.commit()
    return True

When you run the script, the agent will fetch the diff, notice the SQL injection vulnerability in the new delete_user function, compare it against the safer parameterized query in update_email, and produce a detailed review. The output will look something like:

[CRITICAL]: SQL Injection Vulnerability in delete_user
File: src/api/handlers.py
Line: 16-18
Issue: The delete_user function constructs SQL queries using string formatting
with unsanitized user input. This allows an attacker to inject arbitrary SQL
commands through the user_id parameter. The same vulnerability exists in
get_user on line 10.
Suggestion: Use parameterized queries consistently. Replace:
  query = f"DELETE FROM users WHERE id = '{user_id}'"
  cursor.execute(query)
With:
  cursor.execute("DELETE FROM users WHERE id = ?", (user_id,))
Also fix get_user similarly.

[HIGH]: Type Inconsistency in user_id Parameter
File: src/api/handlers.py
Line: 14 (delete_user), Line: 9 (get_user)
Issue: get_user expects user_id as int, but delete_user accepts it as str.
This inconsistency can lead to confusion and type-related bugs.
Suggestion: Standardize on int for user_id across all functions.

[MEDIUM]: Missing Input Validation
File: src/api/handlers.py
Line: 14
Issue: No validation that user_id is a valid positive integer before
proceeding with deletion.
Suggestion: Add validation at function entry:
  if not isinstance(user_id, int) or user_id <= 0:
      raise ValueError("Invalid user_id")

[LOW]: Missing Docstring Detail
File: src/api/handlers.py
Line: 14
Issue: The docstring for delete_user lacks information about parameters,
return value semantics, and potential exceptions.
Suggestion: Expand docstring to include Args, Returns, and Raises sections.

REVIEW_COMPLETE

8. Adding Multi-Agent Collaboration

One of AutoGen's strongest features is the ability to create agent teams that collaborate. Let's extend the system with a specialized security agent and a refactoring agent. The review agent can delegate specialized checks to these sub-agents and incorporate their findings.

# ========== Specialized Agents ==========

SECURITY_AUDITOR_PROMPT = """You are a specialized application security auditor.
Your ONLY task is to find security vulnerabilities in code.
Focus on: injection attacks, XSS, broken access control, sensitive data exposure,
XML external entities, insecure deserialization, using components with known vulnerabilities,
insufficient logging/monitoring.
Report findings in the standard format. If none found, state that explicitly.
End with SECURITY_AUDIT_COMPLETE."""

REFACTORING_ADVISOR_PROMPT = """You are a code refactoring specialist.
Your ONLY task is to suggest structural improvements for maintainability.
Focus on: extracting methods, reducing complexity, improving naming,
applying design patterns, removing duplication, improving cohesion.
Do NOT report security issues or bugs — those are handled by other agents.
Report findings in the standard format. If none found, state that explicitly.
End with REFACTORING_COMPLETE."""

security_auditor = AssistantAgent(
    name="SecurityAuditor",
    system_message=SECURITY_AUDITOR_PROMPT,
    llm_config={"config_list": config_list, "temperature": 0.1},
    human_input_mode="NEVER",
)

refactoring_advisor = AssistantAgent(
    name="RefactoringAdvisor",
    system_message=REFACTORING_ADVISOR_PROMPT,
    llm_config={"config_list": config_list, "temperature": 0.3},
    human_input_mode="NEVER",
)

# Register the same tools for these agents
for agent in [security_auditor, refactoring_advisor]:
    register_function(read_file, caller=agent, executor=user_proxy, name="read_file",
                     description="Read the contents of a source code file.")
    register_function(read_diff, caller=agent, executor=user_proxy, name="read_diff",
                     description="Fetch the current diff/patch to review.")

Now modify the reviewer's system prompt to include delegation instructions:

# Add to the CODE_REVIEWER_SYSTEM_PROMPT before the output format section:
DELEGATION_INSTRUCTIONS = """
## Delegation Protocol
After your initial review, delegate specialized checks:
1. Ask the SecurityAuditor agent: "Please perform a security audit on the code in this diff."
2. Ask the RefactoringAdvisor agent: "Please suggest refactoring improvements for this code."
Incorporate their findings into your final review summary.
"""

Then set up a group chat where all agents can communicate:

from autogen import GroupChat, GroupChatManager

def create_review_team(config_list):
    """Create a collaborative review team with multiple specialized agents."""
    
    # Create all agents (reviewer, security_auditor, refactoring_advisor, user_proxy)
    reviewer = AssistantAgent(
        name="CodeReviewBot",
        system_message=CODE_REVIEWER_SYSTEM_PROMPT + DELEGATION_INSTRUCTIONS,
        llm_config={"config_list": config_list, "temperature": 0.1},
    )
    
    security_auditor = AssistantAgent(
        name="SecurityAuditor",
        system_message=SECURITY_AUDITOR_PROMPT,
        llm_config={"config_list": config_list, "temperature": 0.1},
    )
    
    refactoring_advisor = AssistantAgent(
        name="RefactoringAdvisor",
        system_message=REFACTORING_ADVISOR_PROMPT,
        llm_config={"config_list": config_list, "temperature": 0.3},
    )
    
    user_proxy = UserProxyAgent(
        name="UserProxy",
        human_input_mode="NEVER",
        max_consecutive_auto_reply=15,
        code_execution_config=False,
        llm_config=False,
    )
    
    # Register tools for all LLM agents
    for agent in [reviewer, security_auditor, refactoring_advisor]:
        register_function(read_file, caller=agent, executor=user_proxy,
                         name="read_file",
                         description="Read the contents of a source code file.")
        register_function(read_diff, caller=agent, executor=user_proxy,
                         name="read_diff",
                         description="Fetch the current diff/patch to review.")
    
    # Create group chat
    group_chat = GroupChat(
        agents=[user_proxy, reviewer, security_auditor, refactoring_advisor],
        messages=[],
        max_round=15,
        speaker_selection_method="auto",  # Let AutoGen decide who speaks next
        allow_repeat_speaker=True,
    )
    
    manager = GroupChatManager(
        groupchat=group_chat,
        llm_config={"config_list": config_list},
    )
    
    return group_chat, manager

# Usage
group_chat, manager = create_review_team(config_list)
user_proxy.initiate_chat(
    manager,
    message="CodeReviewBot, please lead a thorough review of the current diff. "
            "Delegate to SecurityAuditor and RefactoringAdvisor as needed.",
    max_turns=20,
)

Best Practices for Code Review Agents

Keep the System Prompt Specific and Measurable

A vague system prompt produces vague reviews. Define concrete criteria with examples. Instead of "check for bad code," specify "check for SQL injection via string formatting, missing parameterization, and dynamic query construction." The LLM performs dramatically better when given explicit, enumerable checks.

Use Low Temperature for Consistency

Set temperature between 0.0 and 0.2 for review agents. Higher temperatures introduce creative variation that's undesirable for analytical tasks — you don't want the agent to sometimes flag an issue and sometimes overlook it based on sampling randomness.

Implement a Severity Taxonomy

The CRITICAL/HIGH/MEDIUM/LOW taxonomy helps developers triage feedback. Without it, a minor naming nitpick carries the same weight as a security vulnerability. Teach the agent to categorize every finding and optionally configure it to skip LOW-severity issues if the diff is large.

Ground Reviews in Actual Diffs, Not Speculation

Always provide the agent with the actual code changes (via diff or full file access). Agents hallucinate convincingly when asked to review code they haven't seen. The tool-use pattern shown above — where the agent calls read_diff and read_file — ensures it works from real data.

Set Max Turns and Termination Conditions

Without termination logic, conversations can loop indefinitely. Use max_turns, is_termination_msg, or a custom termination function. The REVIEW_COMPLETE marker pattern shown earlier gives the agent a clear signal for when to stop.

Log Everything for Continuous Improvement

AutoGen provides full conversation transcripts. Store these logs and periodically review them to refine your system prompts. You'll notice patterns — maybe the agent consistently misses async/await issues or over-flags certain patterns — and you can address these by iterating on the prompt.

Combine with Traditional Static Analysis

LLM-based review complements, not replaces, traditional tools. Run pylint, mypy, bandit, or semgrep before the AI review and include their output in the context. This frees the LLM to focus on higher-level reasoning while the deterministic tools handle straightforward rule violations.

Handle Large Diffs with Chunking

For diffs exceeding the model's context window, implement a chunking strategy. Split the diff into manageable pieces (e.g., 200 lines each), review each chunk independently, then have the agent synthesize a consolidated report. AutoGen's multi-turn nature makes this straightforward — you can iterate through chunks in a single conversation session.

Respect Rate Limits and Costs

Code review agents consume tokens proportional to diff size and review depth. For a typical 200-line diff with full context, expect 3,000-8,000 tokens consumed. At scale (hundreds of PRs daily), consider using smaller, fine-tuned models for initial triage and reserving GPT-4-class models for high-risk changes only.

Conclusion

Building a code review agent with AutoGen transforms a traditionally manual, bottleneck-prone process into an automated, scalable, and consistent quality gate. The framework's conversation-driven architecture lets you move beyond simple prompt-response patterns into rich, multi-agent collaborations where specialized agents handle security, performance, and maintainability concerns in parallel. By defining a clear persona, registering file-access tools, and orchestrating conversations with proper termination conditions, you create a review system that's both powerful and controllable. The techniques covered here — structured system prompts, severity taxonomies, tool integration, group chat delegation, and logging for continuous improvement — form a solid foundation that you can adapt to any language ecosystem, review workflow, or team standard. Start with the single-agent pattern, validate its output against your team's expectations, then expand into multi-agent collaboration as you gain confidence in the system's reliability and cost profile.