Agent Review Gate: a Schema-Forced Approve or Block in CI

The Stack

Tested Against

Prerequisites

• Claude Code (claude -p) or Codex (codex exec)
• A CI job that runs on each change and can fail on a non-zero exit
• An API key in CI: ANTHROPIC_API_KEY for Claude Code, CODEX_API_KEY for Codex

Steps

1. 1

   Force a machine-readable verdict with a schema

   Run the agent read-only over the diff and force a structured answer instead of prose. Claude Code: --output-format json with --json-schema, then read .structured_output.decision with jq. Codex: --output-schema review-schema.json with -o verdict.json (it runs read-only by default, exactly the least privilege you want for a gatekeeper). Your script exits non-zero on block.

2. 2

   What CI checks: the schema enforces approve/block and the gate blocks correctly

   CI validates the review schema (it must enforce an approve/block enum and require a decision plus a reason) and the gate logic (decision=block exits non-zero to fail the merge; decision=approve passes). That is the deterministic contract between the model's verdict and your pipeline. No model, no key.

   cat > review-schema.json <<'EOF'
   {
     "type": "object",
     "properties": {
       "decision": { "type": "string", "enum": ["approve", "block"] },
       "reason": { "type": "string" }
     },
     "required": ["decision", "reason"],
     "additionalProperties": false
   }
   EOF
   cat > checkgate.mjs <<'EOF'
   import { readFileSync } from 'node:fs';
   const schema = JSON.parse(readFileSync('review-schema.json', 'utf8'));
   let ok = true;
   function check(label, cond){ console.log(label + ': ' + (cond ? 'yes' : 'NO')); if(!cond) ok=false; }
   const dec = schema.properties && schema.properties.decision;
   check('schema enforces approve/block enum', !!dec && Array.isArray(dec.enum) && dec.enum.includes('approve') && dec.enum.includes('block'));
   check('schema requires decision and reason', Array.isArray(schema.required) && schema.required.includes('decision') && schema.required.includes('reason'));
   // the gate logic from the recipe: block -> non-zero (fail merge), approve -> zero
   function gate(verdict){ return verdict.decision === 'block' ? 1 : 0; }
   check('gate blocks on decision=block', gate({ decision: 'block', reason: 'unsafe' }) === 1);
   check('gate approves on decision=approve', gate({ decision: 'approve', reason: 'ok' }) === 0);
   if(!ok){ console.log('review gate check FAILED'); process.exit(1); }
   console.log('review gate check OK');
   EOF
   node checkgate.mjs

3. 3

   Drop it in CI (the model review step, not checked by CI)

   Wire the gate into your pipeline so a real diff gets reviewed on every change. The review itself runs a model and is non-deterministic, so it is fenced. CI proves the schema is strict and the gate fails the merge on block, never the model's judgement of a specific diff.