The Reviewer That Reviewed Itself

The pull request that deployed our AI code review workflow got reviewed by the AI code review workflow.

We didn’t plan it that way. The timing just aligned: the reviewer was already live in a shared workflows monorepo when we opened the service-repo PR to wire it up. The reviewer ran automatically. The code it was analyzing was the caller that invoked it — a loop so clean it almost looked staged.

It wasn’t staged. And what it found wasn’t trivial.

---

Two Workflows, One Principle

The system has a clean split at the top level.

The reviewer runs automatically on every pull request. Read-only. Structured scoring across four dimensions — architecture, test coverage, security, maintainability — each rated 1–10 with specific findings. It checks out the PR head, analyzes the diff, and posts a comment. No write access. No secrets beyond what the review itself needs.

The assistant runs on demand. Mention @claude in a PR comment and it reads the thread, analyzes the code, and can push fixes directly. Write-enabled. This is the powerful one — and the dangerous one if not carefully scoped. The workflow structure is nearly identical to the reviewer: same reusable pattern, same thin caller. The differences are the trigger (issue_comment instead of pull_request), a freeform prompt instead of a structured scoring template, and write permissions instead of read-only.

Both live as reusable workflows in a centralized monorepo. Each service repo has a thin caller — a few lines that invoke the shared workflow and point to a repo-specific prompt file. The prompt carries team conventions: patterns to enforce, test coverage minimums, libraries in use. If the caller repo has no prompt file, the workflow fetches a default. Simple, composable, consistent.

The reusable workflow in the central monorepo (already including the fixes from rounds one and two — more on those shortly):

# shared-workflows/.github/workflows/ai-code-review.yml
# Prerequisite: ANTHROPIC_API_KEY set as a repository secret.
name: AI Code Review

on:
  workflow_call:
    inputs:
      head_sha:
        required: true
        type: string
      pr_number:
        required: true
        type: string
  pull_request:              # also works standalone, not only as a reusable workflow
    types: [opened, synchronize]

permissions:
  contents: read
  pull-requests: write

concurrency:
  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
  cancel-in-progress: true

jobs:
  claude-code-review:
    runs-on: ubuntu-latest
    timeout-minutes: 30
    steps:
      - name: Resolve PR context
        id: pr-context
        run: |
          # inputs.* set when called via workflow_call
          # github.event.pull_request.* set when triggered directly as pull_request
          HEAD_SHA="${{ inputs.head_sha || github.event.pull_request.head.sha }}"
          PR_NUMBER="${{ inputs.pr_number || github.event.pull_request.number }}"
          if [ -z "$HEAD_SHA" ] || [ -z "$PR_NUMBER" ]; then
            echo "Error: could not resolve PR context. Pass head_sha and pr_number when calling via workflow_call."
            exit 1
          fi
          echo "head_sha=$HEAD_SHA" >> "$GITHUB_OUTPUT"
          echo "pr_number=$PR_NUMBER" >> "$GITHUB_OUTPUT"

      - uses: actions/checkout@v4
        with:
          ref: ${{ steps.pr-context.outputs.head_sha }}  # PR head so Claude can read source files

      - name: Load review prompt
        id: prompt
        run: |
          # Base prompt lives in the shared-workflows repo.
          # Local copy used when this workflow runs on itself; API fetch used from service repos.
          if [ -f .claude/prompts/code-review.md ]; then
            BASE=$(cat .claude/prompts/code-review.md)
          else
            BASE=$(gh api repos/my-org/shared-workflows/contents/.claude/prompts/code-review.md \
              --jq '.content' | base64 -d) || { echo "Error: failed to fetch base prompt"; exit 1; }
          # base64 -d is GNU/Linux (works on ubuntu-latest); macOS uses -D
          fi
          [ -z "$BASE" ] && { echo "Error: base prompt is empty — refusing to run"; exit 1; }

          # Append repo-specific rules if the calling repo provides them
          RULES=""
          [ -f .claude/prompts/code-review-rules.md ] && RULES=$(cat .claude/prompts/code-review-rules.md)

          # Use a random delimiter to avoid collision if the prompt contains a literal "EOF"
          DELIM=$(openssl rand -hex 8)
          echo "PROMPT_CONTENT<<$DELIM" >> $GITHUB_OUTPUT
          printf '%s\n\n%s' "$BASE" "$RULES" >> $GITHUB_OUTPUT
          echo "$DELIM" >> $GITHUB_OUTPUT
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

      - uses: oven-sh/setup-bun@v2  # claude-code-action requires Bun at runtime

      - uses: anthropics/claude-code-action@v1.0.89
        with:
          anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
          github_token: ${{ secrets.GITHUB_TOKEN }}
          prompt: |
            REPO: ${{ github.repository }}
            PR NUMBER: ${{ steps.pr-context.outputs.pr_number }}

            ${{ steps.prompt.outputs.PROMPT_CONTENT }}
          base_branch: main
          claude_args: |
            --allowedTools "Read,Grep,Glob,Bash(git log:*),Bash(gh pr diff:*),mcp__github__get_pull_request,mcp__github__create_and_submit_pull_request_review"

The thin caller in the service repo:

# my-service/.github/workflows/code-review.yml
name: AI Code Review

on:
  pull_request:
    types: [opened, synchronize]

jobs:
  ai-code-review:
    uses: my-org/shared-workflows/.github/workflows/ai-code-review.yml@main
    with:
      head_sha: ${{ github.event.pull_request.head.sha }}
      pr_number: "${{ github.event.pull_request.number }}"
    secrets: inherit

The caller is thirteen lines. All logic lives in the shared workflow. To onboard a new service repo: add the caller, optionally add .claude/prompts/code-review-rules.md with team-specific conventions, done.

The PR we opened added exactly that caller to a frontend service repo. Five minutes of work. The reviewer ran immediately.

---

Round One: The Silent Failure

The reviewer flagged the fallback prompt logic.

If the caller repo had no prompt file, the workflow fetched a default from the central monorepo. But if that fetch returned empty — wrong path, network blip, anything — the workflow continued without error. The AI received an empty prompt string. It produced a review anyway: polite, vague, useless. No error. No indication that something had gone wrong. No way for anyone reading the comment to know the review hadn’t actually run properly.

Fix: validate the prompt before sending it. If it’s empty, fail fast with a descriptive error. The workflow should be loud about not working, not quiet.

A simple oversight, easy to miss when I was focused on the happy path. The workflow would have kept running, Claude would have kept reviewing, and every result would have been quietly wrong.

---

Round Two: The Context That Wasn’t There

Second review, after fixes from round one.

This time the reviewer flagged a structural bug in how the reusable workflow handled GitHub event context.

When a workflow is called via workflow_call — the mechanism that makes it reusable across repos — the calling workflow’s event data doesn’t pass through. github.event_name becomes 'workflow_call', not 'pull_request'. github.event.pull_request is null. Any job condition checking github.event_name == 'pull_request' silently evaluates to false and skips the step.

The PR number and head SHA — needed to post the review comment in the right place — weren’t available inside the reusable workflow. The caller had to pass them explicitly as workflow inputs. The reusable workflow had to accept and use those inputs rather than reading them from the GitHub context it no longer had.

The symptom would have been confusing: the workflow runs, exits successfully, posts nothing. You’d think the review was skipped. You wouldn’t immediately know why.

The reviewer caught this by reading the workflow definition and reasoning about what happens under a workflow_call event — not as a lint check, but as actual comprehension of how GitHub Actions reusable workflows work.

---

Round Three: The Security Hole

Third review, after the context fixes.

The reviewer flagged the assistant workflow.

The assistant — the write-enabled one — was missing a fork guard.

The assistant triggers on @claude mentions in PR comments. It has write access — it can push commits. If it ran on fork PRs without restriction, an attacker could open a fork pull request, drop a @claude comment, and trigger a workflow that checks out their code and runs with write permissions to the target repository. The technical name for this class of attack is pwn-request. It’s well documented in GitHub Actions security literature, and it’s serious.

The fix is a single condition:

if: github.event.pull_request.head.repo.full_name == github.repository

One line. Refuses to run on fork PRs. But until the reviewer flagged it, it wasn’t there.

I’m not sure we would have caught this in human review. The reviewer and the assistant look similar at a glance — both trigger on PR events, both involve Claude. The difference in permission scope is the thing you have to hold in your head simultaneously with the fork scenario to see the risk. The AI reviewer held it.

---

What the Loop Actually Means

The best test of whether a code review workflow works is whether it improves its own code quality. If the reviewer couldn’t find issues in the code that defines how it runs, either that code was flawless (unlikely) or the reviewer wasn’t working (also unlikely here). It found three issues.

When you build AI-assisted pipelines, make the AI the first consumer of its own output. Not as a party trick. As a quality gate. If your code review workflow can catch a security vulnerability in its own deployment PR before any human reads it, the pipeline is already doing more than most human review processes.

One other thing the reviewer started doing that we hadn’t asked for: on routine PRs, it would sometimes flag that another open pull request was touching the same area of the codebase. If both merged, they’d conflict — not at review time, but at QA, where conflicts are expensive to untangle and the two teams involved have already moved on mentally. The reviewer noticed the overlap before either PR merged. The conversation between the teams happened before the code collided, not after.

No one told it to look at other open PRs. That’s just what a thorough reviewer does — not read the current diff in isolation, but consider what else is in flight. The difference is that a human reviewer has to already know about the parallel work to flag it. The reviewer checked.

---

What Didn’t Go Smoothly

One problem the reviewer couldn’t catch: a compatibility regression in the underlying GitHub Action.

A newer version of the claude-code-action had introduced a dependency on Bun — a JavaScript runtime — via oven-sh/setup-bun. In a GitHub Enterprise environment, that setup action calls api.github.com rather than the enterprise GitHub host. Enterprise credentials don’t work there. The action fails with a 401 before it does anything.

The reviewer flagged some configuration concerns around the action version, but it couldn’t identify the upstream dependency issue without access to the action’s source code. We found that one in production. The fix: pin to an older version that predates the Bun dependency.

The loop worked on logic bugs and security blind spots. Environmental incompatibilities that live upstream of your code — those still require actually running the thing to know.

---

The Loop Closes

We didn’t plan for the reviewer to run on its own deployment PR. Next time we’ll do it deliberately.

The fork guard that blocks pwn-request attacks was in the code before any human read the PR. So was the empty prompt validation and the workflow_call context fix. Three things that would have reached production broken — caught in review, by the thing being deployed.

The tools we build are the first things those tools should run on.