Building the Eval Gate: CI for Models, With the Statistics Done Right.

Why most eval setups do not gate anything

Nearly every team we audit has evals. Very few have a gate. The difference: evals produce a number someone looks at, a gate refuses to promote an artifact when the number is bad. Without the refusal, evals are a dashboard, and dashboards do not stop bad models from reaching production on a Friday.

The second problem is subtler and more damaging. Most teams compare two models on 50 examples, see 82% versus 79%, and ship the winner. On 50 examples that difference is indistinguishable from noise. They have built a machine for making random decisions with statistical decoration.

This post is a working gate: the suite structure, the statistics that make comparisons real, and the CI wiring that turns it into a promotion blocker. It assumes a self-hosted or gateway-fronted stack where you control what reaches production.

The four suites

A gate runs four categories, and each has a different failure semantic.

1. Capability (regression). Does the system do its job? Golden examples with known-good outputs, drawn from real traffic and real edge cases. Failure semantic: score must not drop more than X below the incumbent.

2. Safety and adversarial. Prompt injection attempts, jailbreaks, out-of-scope requests, training-data extraction probes if you fine-tuned. Failure semantic: absolute floor. No regression tolerance, no averaging with capability scores.

3. Format and contract. Valid JSON, required fields, schema conformance, tool-call correctness. Deterministic checks. Failure semantic: near-perfect or fail; downstream code depends on structure, and a 2% malformed rate is an outage at volume.

4. Cost and latency. Tokens per request, p95 latency. Failure semantic: budget ceiling. A model that is one point better and twice the cost is a business decision, not an automatic promotion.

Keeping these separate is the design decision that matters most. A single blended "quality score" lets a capability gain hide a safety regression, which is exactly the failure the gate exists to prevent.

Building the golden set

The set is the asset; the harness is plumbing. Rules that hold up:

• Mine production traffic, do not invent examples. Sample across the real input distribution, and oversample the hard tail.
• Every incident becomes a case. Something goes wrong in production, it enters the golden set with the correct output. The suite grows into an institutional memory of your failure modes.
• 150 to 300 cases minimum for a suite you intend to make promotion decisions with. Fewer, and the confidence intervals below will be too wide to distinguish anything.
• Version and freeze it. The set is a controlled artifact with an owner and a change log. Silently editing it invalidates every historical comparison, and in a regulated environment it invalidates your validation evidence.
• Hold out a private slice that never appears in prompts, few-shot examples, or fine-tuning data. This is your check against having optimized directly for the test.

Grade deterministically wherever possible: exact match, numeric tolerance, schema validation, regex, unit tests on generated code. Reach for an LLM judge only for genuinely open-ended output, and when you do, validate the judge against human labels on a sample first, use a different model family than the one under test, and pin the judge model and prompt as versioned artifacts. An unpinned judge silently redefines your standard every time the vendor updates it.

The statistics, which are not optional

Here is the part that separates a real gate from theater. Two models, 200 examples, 84% versus 81%. Is that a real difference?

Bootstrap resampling answers it in about ten lines. Resample the per-example results with replacement thousands of times, and look at the distribution of the difference:

import numpy as np

def bootstrap_delta(scores_a, scores_b, n=10_000, seed=0):
    """Confidence interval on (A − B), paired by example."""
    rng = np.random.default_rng(seed)
    a, b = np.asarray(scores_a, float), np.asarray(scores_b, float)
    assert a.shape == b.shape, "paired comparison requires identical cases"
    idx = rng.integers(0, len(a), size=(n, len(a)))
    deltas = a[idx].mean(axis=1) - b[idx].mean(axis=1)
    lo, hi = np.percentile(deltas, [2.5, 97.5])
    return {
        "delta": float(a.mean() - b.mean()),
        "ci95": (float(lo), float(hi)),
        "significant": lo > 0 or hi < 0,   # CI excludes zero
        "p_better": float((deltas > 0).mean()),
    }

Run it on the 84-versus-81 case with 200 examples and the 95% interval typically spans roughly −4 to +10 points. The interval includes zero: you cannot distinguish these models. Shipping the "winner" is a coin flip dressed up as engineering.

Two further points that trip teams up. Pair your comparisons: run both models on the identical cases and compare per-example, which removes case difficulty as a variance source and tightens intervals substantially. And if you evaluate ten metrics, expect one to look "significant" at p < 0.05 by chance; either designate one primary metric in advance or apply a multiple-comparison correction.

Non-determinism is its own variance source. Run each case three to five times at your production temperature and average. If a single case flips between runs, that instability is a finding about your system, not noise to smooth away.

The gate

from dataclasses import dataclass

@dataclass
class GateResult:
    passed: bool
    reasons: list[str]

def evaluate_gate(candidate, incumbent, budget) -> GateResult:
    reasons = []

    # 1. Safety: absolute floor, never traded against capability.
    if candidate.safety < 0.98:
        reasons.append(f"SAFETY FLOOR: {candidate.safety:.3f} < 0.980")
    for probe, hit in candidate.critical_probes.items():
        if hit:                      # injection success, data extraction, etc.
            reasons.append(f"CRITICAL PROBE FIRED: {probe}")

    # 2. Contract: structure is binary in practice.
    if candidate.schema_valid < 0.999:
        reasons.append(f"SCHEMA: {candidate.schema_valid:.4f} < 0.9990")

    # 3. Capability: regression must be statistically real AND material.
    cmp = bootstrap_delta(candidate.per_case, incumbent.per_case)
    if cmp["significant"] and cmp["delta"] < -0.02:
        lo, hi = cmp["ci95"]
        reasons.append(
            f"CAPABILITY REGRESSION: {cmp['delta']:+.3f} "
            f"(95% CI {lo:+.3f}..{hi:+.3f})"
        )

    # 4. Budget.
    if candidate.p95_latency_ms > budget.p95_ms:
        reasons.append(f"LATENCY: {candidate.p95_latency_ms}ms > {budget.p95_ms}ms")
    if candidate.cost_per_1k > budget.cost_per_1k:
        reasons.append(f"COST: ${candidate.cost_per_1k:.4f} > ${budget.cost_per_1k:.4f}")

    return GateResult(passed=not reasons, reasons=reasons)

The capability rule deserves a note: it blocks only when a regression is both statistically real and materially large. Without the significance test you block on noise and the team learns to bypass the gate. Without the materiality threshold you block on a half-point that nobody can feel. Both failure modes end the same way, with someone adding a skip flag.

Wiring it into CI

The gate runs on any change to the things that affect behavior: model artifact, prompts, retrieval configuration, tool definitions, inference parameters. Prompts are code; they belong in version control and they trigger the pipeline.

name: model-promotion-gate
on:
  pull_request:
    paths: ["prompts/**", "models/manifest.yaml", "tools/**", "serving/**"]

jobs:
  gate:
    runs-on: [self-hosted, gpu] # evals run inside the boundary
    steps:
      - uses: actions/checkout@v4
      - name: Resolve incumbent baseline
        run: python -m evals.baseline --alias internal-general-v3 --out baseline.json
      - name: Run suites
        run: |
          python -m evals.run --suite capability --repeats 3 --out cand.json
          python -m evals.run --suite safety     --repeats 3 --append cand.json
          python -m evals.run --suite contract   --repeats 1 --append cand.json
      - name: Gate
        run: python -m evals.gate --candidate cand.json --baseline baseline.json
      - uses: actions/upload-artifact@v4
        if: always()
        with: { name: eval-report, path: reports/ }

Two details with outsized value. Run evals inside your boundary, on self-hosted runners, because the golden set contains real production data and does not belong on a public runner. Always upload the report, including on failure, because the artifact is your evidence trail: an immutable record of which model version passed which suite on which date, which is exactly what a validator under SR 11-7 or an ISO 42001 auditor asks for. The gate stops being pure engineering overhead the first time it produces your compliance evidence for free.

Production monitoring closes the loop

Offline evals measure a frozen snapshot. Production drifts: inputs shift, retrieval corpora change, users find new phrasings. Sample a small percentage of live traffic daily, grade it with the same rubric, and alert on movement. When live scores diverge from offline scores, your golden set has gone stale, and refreshing it is the highest-value maintenance work available.

Start here

You do not need the full apparatus on day one. The minimum viable version that actually gates: 50 real capability cases, 20 adversarial probes, deterministic schema checks, a hard safety floor, and CI that fails the build. Add the bootstrap the first time someone proposes shipping on a three-point difference, and grow the golden set from incidents.

The measure of whether you have a gate rather than a dashboard is simple: has it ever blocked a release? If not, either your team is extraordinary or the gate is decorative. It is usually the second one.