The certificates, written out
Each closed form below is checked against the value the study recorded when this page is built. A formula that stops reproducing its own published number fails the build rather than rendering.
Selective-risk certificate
With zero observed violations, split δ across the fixed threshold grid and invert the one-sided exact binomial bound.
Prescribed-prefix ablation: δ=0.1, K=11, n=92 → 0.0498, inside the registered α=.05. The composite artifact refuses four groups, so n falls to 88 and the same formula gives 0.0520 — which only clears α=.10.
Coverage is what pays for it
Risk and coverage trade off exactly as the finite-sample calculation requires; refusing groups is what pushes the bound out.
The composite gate is the only one that refuses any calibration group: 88/92 admitted, 95.7% coverage. Dropping below 92 zero-violation groups is precisely why a 5% bound is no longer available to it.
Preservation is bounded, not proven
No compiled-only failure across the pooled held-out records still leaves a one-sided upper bound on the discordance rate.
Pooling all 90 paired records bounds compiled-only degradation at 3.3%. Any single 30-record family bounds it only at 9.5%. Zero observed failures is not an equivalence result.
Discovery has to amortize
Learning the program costs paid provider episodes, repaid only out of the per-episode saving it later produces.
Issue-type routing spent $0.1146 over 132 discovery episodes and saves $0.000279 per episode → 411 episodes, more than three times its own discovery cohort.
Risk is per artifact, not per paper
4 of the 7 admitted artifacts sit at the registered α=.05. The guarded composite artifact does not: it is the only gate that refuses any calibration group, and refusing four drops n below what a zero-violation 5% bound needs. Every result that rests on it — the composite study and the fair-placement comparator — is a 10%-selective-risk result and is reported as one.
| Admitted artifact | α | Groups admitted | Coverage | Risk bound | Against registered α=.05 |
|---|---|---|---|---|---|
| Prescribed-prefix ablation | 0.05 | 92 / 92 | 100.0% | 0.0498 | meets .05 |
| Natural-order, three-read | 0.10 | 45 / 45 | 100.0% | 0.0992 | licensed at .10 |
| Expanded replication (issue type) | 0.05 | 92 / 92 | 100.0% | 0.0498 | meets .05 |
| PR-outcome audit | 0.05 | 92 / 92 | 100.0% | 0.0498 | meets .05 |
| Backlog-attention routing | 0.05 | 92 / 92 | 100.0% | 0.0498 | meets .05 |
| Guarded composite synthesis | 0.10 | 88 / 92 | 95.7% | 0.0520 | licensed at .10 |
| Comparator deployment (same GCS artifact) | 0.10 | 88 / 92 | 95.7% | 0.0520 | licensed at .10 |
Read this before quoting a number. Three risk levels appear in this work and each licenses a different result set. The three primary workflow families are licensed at .05; the composite and comparator results are licensed at .10 and would retire at .05.
What the primary result establishes
Compiled programs preserve exact outcomes across three distinct decisions and tool vocabularies while removing model turns. Hand-written programs reach the same exact score, so the claim is automatic discovery and lifecycle — not runtime dominance.
| Workflow family | Exact: baseline → compiled | Manual | Requests | Tokens | Cost |
|---|---|---|---|---|---|
| Issue-type routing | 30/30 → 30/30 | 30/30 | −50.0% | −39.5% | −32.0% |
| PR-outcome audit | 30/30 → 30/30 | 30/30 | −75.0% | −80.8% | −75.3% |
| Backlog-attention routing | 29/30 → 30/30 | 30/30 | −74.8% | −81.4% | −75.1% |
| Weighted total | 89/90 → 90/90 | 90/90 | −66.6% | −63.1% | −58.7% |
All records come from one revision-pinned repository snapshot; this is workflow-family, not cross-repository or time-forward, generalization. With no compiled-only failure in 90 paired records, the pooled discordance bound is 3.3%; exact McNemar gives p=1. See the full experimental design for denominators and counterbalancing.
Where the cost numbers come from
Token savings and dollar savings diverge, and the reason was recorded all along. Collapsing three reads into one call shortens the prompt and simultaneously destroys the repeated prefix that made the remaining input cheap.
| Family | Arm | Tokens / episode | Cached | Cost / episode | Break-even |
|---|---|---|---|---|---|
| Issue-type routing | Unchanged | 4,259 | 32.3% | $0.000872 | 411 |
| Compiled | 2,576 | 27.8% | $0.000593 | ||
| Manual macro | 1,781 | 0.0% | $0.000545 | ||
| PR-outcome audit | Unchanged | 2,718 | 0.0% | $0.000679 | 182 |
| Compiled | 522 | 0.0% | $0.000168 | ||
| Manual macro | 522 | 0.0% | $0.000169 | ||
| Backlog-attention routing | Unchanged | 2,845 | 0.0% | $0.000729 | 181 |
| Compiled | 530 | 0.0% | $0.000182 | ||
| Manual macro | 535 | 0.0% | $0.000188 |
In the issue-type family the hand-written macro uses 30.9% fewer total tokens than the compiled condition but is only 8.0% cheaper, because it retains no cache reads at all. The two newer families are cache-cold in every arm, so part of the width of the reported cost range is a property of cache warmth rather than of the compiled programs. Provider-side break-even runs 181–411 episodes. A cache-controlled replication is the correct fix, and this work does not have one.
Generated, not transcribed. Every figure
on this page is read from paper/results/ at build time by
scripts/build_paper_page.py, which re-derives each bound from its
closed form and fails the build on a mismatch. Full derivations, related work, and
threat analysis are in the
paper; the
boundaries are enumerated under
limits.