← Back to DevBytes
Developer Tutorial August 17, 2026 13 min read

Building a Code Review Agent with llama.cpp: Complete Guide

Introduction to Building a Code Review Agent with llama.cpp

Code review is one of the most time-consuming yet critical activities in software development. With the rise of local large language models (LLMs), developers can now build autonomous code review agents that run entirely on their own hardware—no API keys, no data leaving the machine, and no per-token costs. llama.cpp is the leading C++ inference engine for running quantized LLMs efficiently on consumer hardware, making it the perfect foundation for a code review agent.

In this tutorial, you'll learn how to build a complete code review agent using llama.cpp and its Python bindings. The agent will analyze source files, identify potential bugs, suggest improvements, and produce structured review reports—all running locally.

What Is a Code Review Agent?

A code review agent is an automated system that examines source code and provides feedback similar to what a human reviewer would offer. It typically performs:

By leveraging an LLM through llama.cpp, the agent can understand code semantics beyond what traditional linters provide, offering contextual suggestions that consider the intent and logic of the code.

Why llama.cpp?

There are several compelling reasons to choose llama.cpp for this project:

Setting Up Your Environment

Before building the agent, you need to install llama.cpp and download a suitable model. We'll use the Python bindings (llama-cpp-python) for easier integration.

Installing llama-cpp-python

For a CPU-only installation, use pip directly:

pip install llama-cpp-python

If you have an NVIDIA GPU and want CUDA acceleration, install with the appropriate build flags:

CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python --upgrade --force-reinstall --no-cache-dir

For Apple Silicon (M1/M2/M3/M4), Metal acceleration is enabled by default on recent versions, but you can be explicit:

CMAKE_ARGS="-DGGML_METAL=on" pip install llama-cpp-python --upgrade --force-reinstall --no-cache-dir

Verify the installation works:

python -c "from llama_cpp import Llama; print('llama-cpp-python installed successfully')"

Downloading a Model

For code review tasks, you want a model that excels at code understanding. Good options include:

Download a quantized GGUF model from Hugging Face. For this tutorial, we'll use Qwen2.5-Coder-7B in Q4_K_M quantization, which offers a good balance of quality and performance:

# Using huggingface-cli
pip install huggingface-hub
huggingface-cli download Qwen/Qwen2.5-Coder-7B-Instruct-GGUF qwen2.5-coder-7b-instruct-q4_k_m.gguf --local-dir ./models

Alternatively, you can download directly with Python:

from huggingface_hub import hf_hub_download

model_path = hf_hub_download(
    repo_id="Qwen/Qwen2.5-Coder-7B-Instruct-GGUF",
    filename="qwen2.5-coder-7b-instruct-q4_k_m.gguf",
    local_dir="./models"
)
print(f"Model downloaded to: {model_path}")

Building the Code Review Agent

Now let's build the actual agent. We'll structure it as a Python class that handles model loading, prompt construction, file reading, and review generation.

Core Architecture

The agent will consist of several components:

Implementing the Agent

Create a file called code_review_agent.py and add the following implementation:

import os
import json
from pathlib import Path
from typing import Optional
from llama_cpp import Llama


class CodeReviewAgent:
    """A local code review agent powered by llama.cpp."""

    SYSTEM_PROMPT = """You are an expert code reviewer with deep knowledge of software engineering best practices, security, and performance optimization.

Your task is to review the provided code and provide a thorough, structured analysis.

Always format your response as valid JSON with the following structure:
{
  "summary": "Brief overall assessment of the code",
  "issues": [
    {
      "severity": "critical|high|medium|low|info",
      "type": "bug|security|performance|style|maintainability",
      "line": "approximate line number or range, or null",
      "description": "detailed description of the issue",
      "suggestion": "specific fix or improvement recommendation"
    }
  ],
  "positive_aspects": ["list of things done well"],
  "overall_score": "number from 1 to 10"
}

Be precise and actionable. If there are no issues, return an empty issues array. Only report real issues—do not invent problems."""

    def __init__(
        self,
        model_path: str,
        n_ctx: int = 8192,
        n_gpu_layers: int = -1,
        n_threads: Optional[int] = None,
        verbose: bool = False,
    ):
        """Initialize the code review agent with a llama.cpp model.

        Args:
            model_path: Path to the GGUF model file.
            n_ctx: Context window size in tokens.
            n_gpu_layers: Number of layers to offload to GPU (-1 for all).
            n_threads: Number of CPU threads (None for auto).
            verbose: Whether to print llama.cpp internal logs.
        """
        self.model_path = model_path
        self.n_ctx = n_ctx
        self.llm = Llama(
            model_path=model_path,
            n_ctx=n_ctx,
            n_gpu_layers=n_gpu_layers,
            n_threads=n_threads,
            verbose=verbose,
        )

    def _build_review_prompt(self, file_path: str, code: str) -> str:
        """Build the user prompt for reviewing a specific file."""
        file_ext = Path(file_path).suffix or "unknown"
        return f"""Please review the following code file.

File: {file_path}
Language: {file_ext}

{file_ext}
{code}
Analyze this code for bugs, security issues, performance problems, style issues, and maintainability concerns. Return your analysis as JSON."""

    def _extract_json(self, text: str) -> dict:
        """Attempt to extract a JSON object from the model's response."""
        # Try to find JSON between curly braces
        start = text.find("{")
        end = text.rfind("}")
        if start != -1 and end != -1 and end > start:
            json_str = text[start : end + 1]
            try:
                return json.loads(json_str)
            except json.JSONDecodeError:
                pass
        return {"raw_response": text}

    def review_code(
        self,
        code: str,
        file_path: str = "snippet.py",
        max_tokens: int = 2048,
        temperature: float = 0.1,
    ) -> dict:
        """Review a code string and return structured feedback.

        Args:
            code: The source code to review.
            file_path: Virtual file path for context.
            max_tokens: Maximum tokens to generate.
            temperature: Sampling temperature (lower = more focused).

        Returns:
            Dictionary containing the review results.
        """
        user_prompt = self._build_review_prompt(file_path, code)

        response = self.llm.create_chat_completion(
            messages=[
                {"role": "system", "content": self.SYSTEM_PROMPT},
                {"role": "user", "content": user_prompt},
            ],
            max_tokens=max_tokens,
            temperature=temperature,
            top_p=0.9,
        )

        raw_output = response["choices"][0]["message"]["content"]
        review = self._extract_json(raw_output)
        review["model"] = os.path.basename(self.model_path)
        review["file"] = file_path
        return review

    def review_file(self, file_path: str, max_tokens: int = 2048) -> dict:
        """Read a file from disk and review it.

        Args:
            file_path: Path to the source file.
            max_tokens: Maximum tokens to generate.

        Returns:
            Dictionary containing the review results.
        """
        path = Path(file_path)
        if not path.exists():
            raise FileNotFoundError(f"File not found: {file_path}")

        code = path.read_text(encoding="utf-8")
        return self.review_code(code, file_path=str(path), max_tokens=max_tokens)

    def review_directory(
        self,
        directory: str,
        extensions: tuple = (".py", ".js", ".ts", ".java", ".cpp", ".c", ".go", ".rs"),
        max_tokens: int = 2048,
    ) -> list:
        """Review all source files in a directory.

        Args:
            directory: Path to the directory to scan.
            extensions: Tuple of file extensions to include.
            max_tokens: Maximum tokens per review.

        Returns:
            List of review dictionaries.
        """
        reviews = []
        dir_path = Path(directory)

        for file_path in sorted(dir_path.rglob("*")):
            if file_path.is_file() and file_path.suffix in extensions:
                print(f"Reviewing: {file_path}")
                try:
                    review = self.review_file(str(file_path), max_tokens=max_tokens)
                    reviews.append(review)
                except Exception as e:
                    reviews.append({
                        "file": str(file_path),
                        "error": str(e),
                    })
        return reviews

Using the Agent

Now let's create a script that demonstrates how to use the agent to review code. Create a file called run_review.py:

from code_review_agent import CodeReviewAgent
import json

# Initialize the agent
agent = CodeReviewAgent(
    model_path="./models/qwen2.5-coder-7b-instruct-q4_k_m.gguf",
    n_ctx=8192,
    n_gpu_layers=-1,  # Use GPU for all layers if available
    verbose=False,
)

# Example: Review a code snippet with intentional issues
sample_code = '''import os

def read_user_file(filename):
    """Read a file provided by the user."""
    path = "/data/" + filename
    with open(path, "r") as f:
        return f.read()

def process_items(items):
    results = []
    for i in range(len(items)):
        results.append(items[i] * 2)
    return results

def get_config_value(key, config={}):
    return config.get(key, None)

class DataProcessor:
    def __init__(self):
        self.data = []

    def add(self, item):
        self.data.append(item)

    def process(self):
        total = 0
        for item in self.data:
            total += item
        return total / len(self.data)
'''

# Review the code
review = agent.review_code(sample_code, file_path="example.py")

# Print the structured review
print(json.dumps(review, indent=2))

When you run this script, the agent will analyze the code and return a structured JSON review. The sample code above contains several intentional issues that the agent should identify:

Reviewing Files and Directories

To review actual files on disk, use the review_file and review_directory methods:

from code_review_agent import CodeReviewAgent
import json

agent = CodeReviewAgent(
    model_path="./models/qwen2.5-coder-7b-instruct-q4_k_m.gguf",
    n_ctx=8192,
    n_gpu_layers=-1,
)

# Review a single file
review = agent.review_file("src/main.py")
print(json.dumps(review, indent=2))

# Review an entire project directory
reviews = agent.review_directory("src/", extensions=(".py",))

# Save all reviews to a file
with open("review_report.json", "w") as f:
    json.dump(reviews, f, indent=2)

print(f"Reviewed {len(reviews)} files. Report saved to review_report.json")

Generating Human-Readable Reports

Structured JSON is great for programmatic use, but you'll often want a readable report. Let's add a report formatter to our agent. Create report_formatter.py:

import json
from datetime import datetime
from pathlib import Path


SEVERITY_ICONS = {
    "critical": "[CRITICAL]",
    "high": "[HIGH]",
    "medium": "[MEDIUM]",
    "low": "[LOW]",
    "info": "[INFO]",
}


def format_review_markdown(review: dict) -> str:
    """Convert a single review dictionary to a markdown report."""
    lines = []
    lines.append(f"# Code Review Report")
    lines.append(f"")
    lines.append(f"**File:** `{review.get('file', 'unknown')}`")
    lines.append(f"**Model:** {review.get('model', 'unknown')}")
    lines.append(f"**Generated:** {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}")
    lines.append(f"")

    if "error" in review:
        lines.append(f"**Error:** {review['error']}")
        return "\n".join(lines)

    # Summary
    summary = review.get("summary", "No summary provided.")
    lines.append(f"## Summary")
    lines.append(f"")
    lines.append(f"{summary}")
    lines.append(f"")

    # Overall score
    score = review.get("overall_score", "N/A")
    lines.append(f"**Overall Score:** {score}/10")
    lines.append(f"")

    # Issues
    issues = review.get("issues", [])
    if issues:
        lines.append(f"## Issues Found ({len(issues)})")
        lines.append(f"")
        for i, issue in enumerate(issues, 1):
            severity = issue.get("severity", "info")
            icon = SEVERITY_ICONS.get(severity, "[INFO]")
            issue_type = issue.get("type", "general")
            line_num = issue.get("line", "N/A")
            description = issue.get("description", "No description.")
            suggestion = issue.get("suggestion", "No suggestion provided.")

            lines.append(f"### {i}. {icon} {issue_type.capitalize()} Issue")
            lines.append(f"")
            lines.append(f"**Line:** {line_num}")
            lines.append(f"")
            lines.append(f"**Description:** {description}")
            lines.append(f"")
            lines.append(f"**Suggestion:** {suggestion}")
            lines.append(f"")
    else:
        lines.append(f"## Issues Found")
        lines.append(f"")
        lines.append(f"No issues found. The code looks clean!")
        lines.append(f"")

    # Positive aspects
    positives = review.get("positive_aspects", [])
    if positives:
        lines.append(f"## Positive Aspects")
        lines.append(f"")
        for positive in positives:
            lines.append(f"- {positive}")
        lines.append(f"")

    return "\n".join(lines)


def format_reviews_markdown(reviews: list) -> str:
    """Convert multiple reviews into a combined markdown report."""
    sections = []
    sections.append("# Code Review Summary Report")
    sections.append("")
    sections.append(f"**Files Reviewed:** {len(reviews)}")
    sections.append(f"**Generated:** {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}")
    sections.append("")
    sections.append("---")
    sections.append("")

    for review in reviews:
        sections.append(format_review_markdown(review))
        sections.append("")
        sections.append("---")
        sections.append("")

    return "\n".join(sections)


def save_report(reviews: list, output_path: str = "code_review_report.md"):
    """Save reviews as a markdown report file."""
    report = format_reviews_markdown(reviews)
    Path(output_path).write_text(report, encoding="utf-8")
    print(f"Report saved to {output_path}")


# Example usage
if __name__ == "__main__":
    # Load saved reviews and generate a report
    with open("review_report.json", "r") as f:
        reviews = json.load(f)
    save_report(reviews, "code_review_report.md")

Integrating with Git Diffs

A practical code review agent should focus on changes rather than reviewing entire codebases every time. Here's how to integrate with Git to review only modified files:

import subprocess
from pathlib import Path
from code_review_agent import CodeReviewAgent
from report_formatter import save_report


def get_git_diff_files(branch: str = "main") -> list:
    """Get list of files changed compared to the given branch."""
    result = subprocess.run(
        ["git", "diff", "--name-only", branch],
        capture_output=True,
        text=True,
    )
    return [f for f in result.stdout.strip().split("\n") if f]


def get_git_diff_content(file_path: str, branch: str = "main") -> str:
    """Get the diff content for a specific file."""
    result = subprocess.run(
        ["git", "diff", branch, "--", file_path],
        capture_output=True,
        text=True,
    )
    return result.stdout


def review_git_changes(model_path: str, branch: str = "main"):
    """Review all changed files compared to a branch."""
    agent = CodeReviewAgent(
        model_path=model_path,
        n_ctx=8192,
        n_gpu_layers=-1,
    )

    changed_files = get_git_diff_files(branch)
    print(f"Found {len(changed_files)} changed files compared to {branch}")

    reviews = []
    for file_path in changed_files:
        if not Path(file_path).exists():
            print(f"  Skipping deleted file: {file_path}")
            continue

        print(f"  Reviewing: {file_path}")
        diff_content = get_git_diff_content(file_path, branch)

        # Review the current version of the file
        review = agent.review_file(file_path)
        review["diff"] = diff_content
        reviews.append(review)

    return reviews


if __name__ == "__main__":
    reviews = review_git_changes(
        model_path="./models/qwen2.5-coder-7b-instruct-q4_k_m.gguf",
        branch="main",
    )
    save_report(reviews, "git_review_report.md")

Best Practices

To get the most out of your code review agent, follow these best practices:

Choose the Right Model Size

Model selection directly impacts review quality and performance. For code review tasks:

Start with a 7B model and scale up if you find the reviews lack depth or miss subtle issues.

Optimize Inference Parameters

The temperature and sampling parameters significantly affect review quality. For code review:

Handle Large Files Gracefully

Large source files may exceed the model's context window. Implement chunking to handle this:

def review_large_file(self, file_path: str, chunk_size: int = 200) -> list:
    """Review a large file by splitting it into chunks."""
    path = Path(file_path)
    lines = path.read_text(encoding="utf-8").splitlines()

    reviews = []
    for i in range(0, len(lines), chunk_size):
        chunk = "\n".join(lines[i : i + chunk_size])
        start_line = i + 1
        end_line = min(i + chunk_size, len(lines))

        review = self.review_code(
            chunk,
            file_path=f"{file_path} (lines {start_line}-{end_line})",
        )
        reviews.append(review)

    return reviews

Use Structured Output Consistently

Always instruct the model to return structured JSON. This makes it easy to:

If the model occasionally fails to produce valid JSON, implement a retry mechanism or fall back to regex-based extraction as shown in the _extract_json method.

Cache and Batch Reviews

Running inference is computationally expensive. To avoid redundant work:

Here's a simple caching implementation:

import hashlib
import json
from pathlib import Path


def get_file_hash(file_path: str) -> str:
    """Generate a hash of file contents for cache keys."""
    content = Path(file_path).read_bytes()
    return hashlib.sha256(content).hexdigest()


def get_cached_review(file_path: str, cache_dir: str = ".review_cache") -> dict:
    """Retrieve a cached review if the file hasn't changed."""
    file_hash = get_file_hash(file_path)
    cache_file = Path(cache_dir) / f"{file_hash}.json"
    if cache_file.exists():
        return json.loads(cache_file.read_text())
    return None


def save_cached_review(file_path: str, review: dict, cache_dir: str = ".review_cache"):
    """Save a review to the cache."""
    file_hash = get_file_hash(file_path)
    cache_file = Path(cache_dir) / f"{file_hash}.json"
    Path(cache_dir).mkdir(exist_ok=True)
    cache_file.write_text(json.dumps(review, indent=2))

Validate Model Suggestions

LLMs can produce plausible but incorrect suggestions. Always treat the agent's output as advisory:

Conclusion

Building a code review agent with llama.cpp gives you a powerful, private, and cost-effective tool for improving code quality. By running entirely locally, the agent can review proprietary code without any data privacy concerns. The structured JSON output makes it easy to integrate into existing development workflows, from pre-commit hooks to CI/CD pipelines. Start with a 7B model like Qwen2.5-Coder, experiment with the inference parameters to match your quality requirements, and scale up to larger models as your hardware allows. Remember that the agent is a complement to human review, not a replacement—use it to catch low-hanging fruit and surface potential issues early, letting human reviewers focus on architecture, business logic, and the nuanced decisions that require human judgment. With the foundation built in this tutorial, you can extend the agent with features like multi-language support, diff-aware reviews, IDE integration, and automated fix suggestions to create a comprehensive local code quality pipeline.

— Ad —

Google AdSense will appear here after approval

← Back to all articles