How to Build a Self-Correcting Agent with Reflection

How to Build a Self-Correcting Agent with Reflection

Large language models are powerful, but they are not infallible. They hallucinate facts, produce buggy code, and occasionally misinterpret instructions. A self-correcting agent with reflection is an architectural pattern that addresses these shortcomings by giving the model the ability to review, critique, and revise its own outputs before returning a final answer. This tutorial walks through the concept, the implementation, and the best practices for building one yourself.

What Is a Reflective Agent?

A reflective agent is an AI agent that follows a loop of generation, evaluation, and revision. Instead of accepting the first response from an LLM, the agent asks the same model (or a second one) to evaluate the response against a set of criteria. If the evaluation surfaces problems, the agent feeds the critique back into the model along with the original output, prompting it to produce an improved version.

The pattern was popularized by research such as the Reflexion paper and has since become a staple in production agent frameworks like LangGraph, AutoGen, and CrewAI. The core insight is simple: models that are asked to critique their own work often catch errors that they missed during initial generation.

Why Reflection Matters

Reflection matters because it converts a single-shot prediction into an iterative refinement process. The benefits include:

• Higher accuracy: Self-critique catches factual and logical errors before they reach the user.
• Better code quality: For coding agents, reflection can simulate running tests, reading tracebacks, and patching bugs.
• Reduced hallucination: Asking the model to justify its claims often exposes unsupported assertions.
• Alignment with instructions: A reviewer prompt can check whether every requirement in the original task was satisfied.
• Graceful degradation: When the model cannot solve a problem, the reflection trace provides diagnostic value for debugging.

The tradeoff is latency and cost. Each reflection cycle adds another LLM call, so the pattern should be applied judiciously rather than to every trivial query.

Architecture of a Self-Correcting Agent

At a high level, the agent has four components:

• Generator: Produces an initial candidate response.
• Evaluator: Inspects the candidate and produces structured feedback.
• Reviser: Takes the original task, the candidate, and the feedback, then produces an improved candidate.
• Controller: Decides when to stop — either because the evaluator is satisfied or a maximum iteration count is reached.

The controller is essential. Without a termination condition, the agent can loop indefinitely, burning tokens while making marginal improvements. A typical configuration allows between two and five reflection cycles.

Implementing the Agent

The following example uses Python with the OpenAI client, but the same structure applies to any LLM provider. We will build a minimal but complete reflective agent that writes a Python function, critiques it, and revises it.

Step 1: Define the LLM Helper

First, create a thin wrapper around the chat completion API so the rest of the code stays clean.

import os
import json
from openai import OpenAI

client = OpenAI(api_key=os.environ.get("OPENAI_API_KEY"))

def call_llm(system_prompt: str, user_prompt: str, model: str = "gpt-4o") -> str:
    """Send a single message to the LLM and return the text response."""
    response = client.chat.completions.create(
        model=model,
        messages=[
            {"role": "system", "content": system_prompt},
            {"role": "user", "content": user_prompt},
        ],
        temperature=0.2,
    )
    return response.choices[0].message.content

We use a low temperature for deterministic, focused outputs. Reflection benefits from consistency — a wildly creative evaluator is harder to reason about than a strict one.

Step 2: Write the Generator

The generator takes a task description and produces an initial solution.

GENERATOR_SYSTEM = """You are an expert Python engineer.
Given a task, write a complete, production-ready Python function.
Return only the code inside a single markdown code block."""

def generate(task: str) -> str:
    return call_llm(GENERATOR_SYSTEM, task)

Step 3: Write the Evaluator

The evaluator is the heart of the reflection loop. We ask it to return structured JSON so the controller can make decisions programmatically.

EVALUATOR_SYSTEM = """You are a strict code reviewer.
Evaluate the provided Python code against the task.
Return JSON with two fields:
- "score": an integer from 0 to 10 indicating quality.
- "feedback": a concise list of issues, or "No issues found." if perfect.
Be rigorous. Check correctness, edge cases, error handling, and style."""

def evaluate(task: str, candidate: str) -> dict:
    user_prompt = f"Task:\n{task}\n\nCode to review:\n{candidate}"
    raw = call_llm(EVALUATOR_SYSTEM, user_prompt)
    try:
        return json.loads(raw)
    except json.JSONDecodeError:
        # Fallback if the model wraps JSON in prose
        start = raw.find("{")
        end = raw.rfind("}") + 1
        return json.loads(raw[start:end])

Notice the fallback parser. LLMs occasionally wrap JSON in markdown fences or explanatory text, so defensive parsing is a practical necessity.

Step 4: Write the Reviser

The reviser receives the original task, the current candidate, and the feedback, then produces an improved version.

REVISER_SYSTEM = """You are an expert Python engineer.
You will receive a task, your previous code, and reviewer feedback.
Produce an improved version that addresses every point in the feedback.
Return only the code inside a single markdown code block."""

def revise(task: str, candidate: str, feedback: str) -> str:
    user_prompt = (
        f"Task:\n{task}\n\n"
        f"Previous code:\n{candidate}\n\n"
        f"Reviewer feedback:\n{feedback}\n\n"
        f"Produce the corrected code."
    )
    return call_llm(REVISER_SYSTEM, user_prompt)

Step 5: Assemble the Reflection Loop

Now we tie the pieces together with a controller that enforces a maximum number of iterations and stops early when the evaluator is satisfied.

def reflective_agent(task: str, max_iterations: int = 3, threshold: int = 9) -> dict:
    """Run a self-correcting agent that generates, evaluates, and revises."""
    candidate = generate(task)
    trace = []

    for i in range(1, max_iterations + 1):
        evaluation = evaluate(task, candidate)
        score = evaluation.get("score", 0)
        feedback = evaluation.get("feedback", "")
        trace.append({"iteration": i, "score": score, "feedback": feedback})

        print(f"Iteration {i}: score={score}")

        if score >= threshold or "No issues" in feedback:
            return {"code": candidate, "trace": trace, "converged": True}

        candidate = revise(task, candidate, feedback)

    # Final evaluation after the last revision
    final_eval = evaluate(task, candidate)
    trace.append({
        "iteration": max_iterations + 1,
        "score": final_eval.get("score", 0),
        "feedback": final_eval.get("feedback", ""),
    })

    return {
        "code": candidate,
        "trace": trace,
        "converged": final_eval.get("score", 0) >= threshold,
    }

Step 6: Run the Agent

Invoke the agent with a concrete task and inspect both the final code and the reflection trace.

if __name__ == "__main__":
    task = (
        "Write a Python function called `safe_divide(a, b)` that divides a by b. "
        "It must handle division by zero by returning None, handle non-numeric "
        "inputs by raising TypeError with a clear message, and include a docstring "
        "with a usage example."
    )

    result = reflective_agent(task, max_iterations=3, threshold=9)

    print("\n=== FINAL CODE ===")
    print(result["code"])
    print("\n=== CONVERGED ===", result["converged"])
    print("\n=== REFLECTION TRACE ===")
    for step in result["trace"]:
        print(f"  Iteration {step['iteration']}: score={step['score']}")
        print(f"    Feedback: {step['feedback']}")

When you run this, you will typically see the first iteration produce a reasonable but imperfect function — perhaps missing the TypeError for non-numeric inputs. The evaluator flags the gap, and the reviser closes it in the next iteration. By the second or third cycle, the code usually converges.

Adding Tool-Based Reflection

Self-critique is powerful, but the model is still reasoning about code without executing it. For coding agents, the most effective reflection comes from real execution feedback. You can extend the evaluator to actually run the candidate code against a set of test cases.

import subprocess
import tempfile

def execute_with_tests(code: str, test_code: str) -> dict:
    """Run the candidate code alongside test cases in a subprocess."""
    full_script = code + "\n\n" + test_code
    with tempfile.NamedTemporaryFile(mode="w", suffix=".py", delete=False) as f:
        f.write(full_script)
        path = f.name

    result = subprocess.run(
        ["python", path],
        capture_output=True,
        text=True,
        timeout=10,
    )
    return {
        "stdout": result.stdout,
        "stderr": result.stderr,
        "returncode": result.returncode,
    }

You can then feed the stderr output directly into the reviser prompt. This transforms the agent from one that speculates about bugs into one that observes them. The loop becomes: generate, run tests, read traceback, patch, repeat.

def reflective_coding_agent(task: str, tests: str, max_iterations: int = 4) -> str:
    candidate = generate(task)

    for i in range(max_iterations):
        execution = execute_with_tests(candidate, tests)
        if execution["returncode"] == 0:
            print(f"All tests passed on iteration {i + 1}.")
            return candidate

        print(f"Iteration {i + 1}: tests failed.")
        candidate = revise(task, candidate, execution["stderr"])

    return candidate

This pattern — sometimes called execution-guided reflection — is what powers many state-of-the-art coding agents. The model no longer has to guess whether its code works; the environment tells it.

Best Practices

Cap the Iteration Count

Always set a hard maximum on reflection cycles. Diminishing returns set in quickly, and an agent stuck in a loop of marginal edits wastes both time and money. Three to five iterations is a sensible default for most tasks.

Use Structured Output for Evaluation

Free-text feedback is hard to act on programmatically. Ask the evaluator to return JSON with a numeric score and a list of specific issues. This lets the controller make clean decisions about whether to continue or stop.

Separate Generator and Evaluator Prompts

The generator and evaluator should have distinct system prompts that encode different priorities. The generator is optimistic and productive; the evaluator is skeptical and rigorous. This role separation produces more useful tension than a single prompt that tries to do both.

Make Feedback Specific and Actionable

Vague feedback like "the code could be better" leads to vague revisions. Instruct the evaluator to cite specific lines, name specific failure modes, and propose specific fixes. The reviser can only act on what the evaluator articulates.

Log the Full Reflection Trace

Persist every iteration — the candidate, the score, and the feedback. This trace is invaluable for debugging failures, auditing agent behavior, and improving your prompts over time. In production, write it to your observability platform alongside latency and token usage.

Choose the Right Model for Each Role

You do not have to use the same model for generation and evaluation. A common strategy is to use a stronger model for evaluation and a faster, cheaper model for generation, or vice versa depending on where quality bottlenecks lie. Experiment with combinations.

Avoid Reflection on Trivial Tasks

Reflection adds latency and cost. For simple lookups, formatting tasks, or single-step questions, a single LLM call is sufficient. Reserve the reflective loop for tasks where correctness genuinely matters — code generation, mathematical reasoning, structured data extraction, and multi-step planning.

Conclusion

A self-correcting agent with reflection turns a one-shot prediction into an iterative refinement process, dramatically improving output quality on complex tasks. By combining a generator, an evaluator, a reviser, and a controller with a sensible termination condition, you get an agent that catches its own mistakes before the user ever sees them. Start with the minimal loop described here, add execution-based feedback where possible, and tune the iteration count and evaluation criteria to your specific domain. The result is an agent that does not just produce answers — it produces answers it has already vetted.