How to Build an AI PR Generator with GitHub Actions

Pull requests are the heartbeat of modern software collaboration, but writing a clear, well-structured PR description is often the last thing developers want to do after hours of coding. An AI PR Generator automates this tedious step by analyzing your code changes and producing a human-readable summary, test plan, and risk assessment automatically. In this tutorial, you'll learn how to build one using GitHub Actions and a large language model API.

What Is an AI PR Generator?

An AI PR Generator is a GitHub Action that triggers whenever a pull request is opened or updated. It inspects the diff between the source and target branches, sends that diff (along with contextual metadata) to an AI model, and posts the generated summary as a comment on the PR. The result is consistent, informative PR descriptions without manual effort.

Why It Matters

Faster reviews: Reviewers get an immediate high-level overview before diving into code.
Consistency: Every PR follows the same structure, making history easier to scan.
Onboarding: New team members understand changes faster with AI-generated context.
Reduced toil: Authors skip the boilerplate of writing "what changed and why."
Quality signal: A model can flag risky changes or missing tests before review.

Prerequisites

Before you start, make sure you have:

A GitHub repository where you have admin access.
An API key for an LLM provider (this tutorial uses OpenAI, but Anthropic or local models work too).
Basic familiarity with YAML and JavaScript.

Step 1: Store Your API Key Securely

Never hardcode secrets in your workflow. Add your OpenAI API key as a repository secret:

Go to Settings → Secrets and variables → Actions in your repository.
Click New repository secret.
Name it OPENAI_API_KEY and paste your key value.

Step 2: Create the GitHub Action Workflow

Create a file at .github/workflows/ai-pr-generator.yml. This workflow listens for pull request events and calls a custom action that we'll build next.

name: AI PR Generator

on:
  pull_request:
    types: [opened, synchronize, reopened]

permissions:
  pull-requests: write
  contents: read

jobs:
  generate:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Generate AI PR description
        uses: ./.github/actions/ai-pr-generator
        with:
          openai-api-key: ${{ secrets.OPENAI_API_KEY }}
          github-token: ${{ secrets.GITHUB_TOKEN }}

The fetch-depth: 0 setting ensures the full git history is available so we can compute accurate diffs. The permissions block grants the action the ability to post comments on the PR.

Step 3: Build the Composite Action

Create a directory .github/actions/ai-pr-generator with an action.yml file. This composite action will install dependencies, run a Node script, and post the result.

name: 'AI PR Generator'
description: 'Generates a PR description using an LLM'
inputs:
  openai-api-key:
    description: 'OpenAI API key'
    required: true
  github-token:
    description: 'GitHub token for API calls'
    required: true

runs:
  using: 'composite'
  steps:
    - name: Setup Node
      uses: actions/setup-node@v4
      with:
        node-version: '20'

    - name: Install dependencies
      shell: bash
      run: |
        cd ${{ github.action_path }}
        npm install

    - name: Run generator
      shell: bash
      env:
        OPENAI_API_KEY: ${{ inputs.openai-api-key }}
        GITHUB_TOKEN: ${{ inputs.github-token }}
        PR_NUMBER: ${{ github.event.pull_request.number }}
        REPO: ${{ github.repository }}
        BASE_SHA: ${{ github.event.pull_request.base.sha }}
        HEAD_SHA: ${{ github.event.pull_request.head.sha }}
      run: node ${{ github.action_path }}/index.js

Step 4: Write the Generator Script

In the same directory, create package.json and index.js. The script fetches the diff, calls the LLM, and posts the comment.

{
  "name": "ai-pr-generator",
  "version": "1.0.0",
  "type": "module",
  "dependencies": {
    "openai": "^4.67.0"
  }
}

import { execSync } from "node:child_process";
import OpenAI from "openai";

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
const GITHUB_TOKEN = process.env.GITHUB_TOKEN;
const REPO = process.env.REPO;
const PR_NUMBER = process.env.PR_NUMBER;
const BASE_SHA = process.env.BASE_SHA;
const HEAD_SHA = process.env.HEAD_SHA;

function getDiff() {
  return execSync(`git diff ${BASE_SHA} ${HEAD_SHA} --stat=200`, {
    encoding: "utf8",
    maxBuffer: 1024 * 1024 * 50,
  });
}

function getFullDiff() {
  return execSync(`git diff ${BASE_SHA} ${HEAD_SHA}`, {
    encoding: "utf8",
    maxBuffer: 1024 * 1024 * 50,
  });
}

function truncate(text, maxChars) {
  if (text.length <= maxChars) return text;
  return text.slice(0, maxChars) + "\n...[truncated]...";
}

async function generateDescription(diffStat, fullDiff) {
  const prompt = `You are a senior software engineer writing a pull request description.
Analyze the following git diff and produce a concise, well-structured PR summary.

## Diff Stat
${diffStat}

## Full Diff
${truncate(fullDiff, 12000)}

Respond in this exact Markdown format:

## Summary
<2-3 sentence overview of what changed and why>

## Changes
- <bullet point>
- <bullet point>

## Risk
<brief note on potential risks or breaking changes>

## Test Plan
- <how to verify this change>
`;

  const response = await openai.chat.completions.create({
    model: "gpt-4o-mini",
    messages: [{ role: "user", content: prompt }],
    temperature: 0.2,
    max_tokens: 800,
  });

  return response.choices[0].message.content.trim();
}

async function postComment(body) {
  const url = `https://api.github.com/repos/${REPO}/issues/${PR_NUMBER}/comments`;
  const res = await fetch(url, {
    method: "POST",
    headers: {
      Authorization: `Bearer ${GITHUB_TOKEN}`,
      Accept: "application/vnd.github+json",
      "X-GitHub-Api-Version": "2022-11-28",
    },
    body: JSON.stringify({ body }),
  });
  if (!res.ok) {
    throw new Error(`GitHub API error: ${res.status} ${await res.text()}`);
  }
}

async function main() {
  console.log("Fetching diff...");
  const diffStat = getDiff();
  const fullDiff = getFullDiff();

  console.log("Generating description with LLM...");
  const description = await generateDescription(diffStat, fullDiff);

  const comment = `### 🤖 AI-Generated PR Description\n\n${description}\n\n---\n_Generated automatically by the AI PR Generator action._`;

  console.log("Posting comment...");
  await postComment(comment);
  console.log("Done.");
}

main().catch((err) => {
  console.error(err);
  process.exit(1);
});

Step 5: Test the Workflow

Commit all files and push to your repository. Then open a new pull request with some code changes. Within a minute, you should see a comment appear on the PR with the AI-generated summary. If something fails, check the Actions tab for logs — common issues include missing secrets, insufficient permissions, or diffs that exceed the model's context window.

Best Practices

Truncate large diffs: Cap the diff size sent to the model to avoid token limits and runaway costs. The script above truncates at 12,000 characters.
Use a cheap, fast model: gpt-4o-mini is ideal for summaries. Reserve larger models for complex refactors.
Idempotency: Before posting, fetch existing comments and update or skip if a previous AI comment exists. This prevents duplicate posts on every push.
Respect file ignore lists: Filter out lock files, generated assets, and package-lock.json from the diff to focus the model on meaningful changes.
Set temperature low: A value around 0.2 keeps output deterministic and factual.
Guard secrets: Always pass keys through secrets and never log them. Use the least-privileged GITHUB_TOKEN scope.
Provide escape hatches: Allow authors to skip generation by including [skip-ai-pr] in the PR title or body.
Cache dependencies: Use actions/cache for node_modules to speed up repeated runs.

Advanced Enhancements

Once the basic flow works, consider extending it:

Update the PR body instead of posting a comment using the PATCH /repos/{owner}/{repo}/pulls/{pull_number} endpoint.
Add labels automatically based on detected change types (e.g., bug, feature, docs).
Generate suggested test cases by asking the model to propose unit tests for the new code.
Use embeddings to pull in related files or past PRs for richer context.
Switch providers by swapping the OpenAI client for Anthropic, Mistral, or a self-hosted endpoint.

Conclusion

Building an AI PR Generator with GitHub Actions is a small investment that pays off every time someone opens a pull request. By combining git diffs, a lightweight Node script, and an LLM API, you can produce consistent, reviewer-friendly descriptions automatically. Start with the minimal workflow above, then layer in best practices like diff truncation, idempotent comments, and label automation to tailor the experience to your team. With a few dozen lines of code, you turn a repetitive chore into a reliable, always-on part of your development workflow.