Guarded Agentic Compaction

Every benchmark in the evaluation, and what each one can prove.

Ten named external benchmarks plus the frozen multidomain record sources. The evaluation deliberately does not average them: a screened reference plan, an executed simulator, a real compiler run, and a source we were denied access to support different claims, so each row keeps its own substrate, denominator, execution status, and boundary.

10named external benchmarks audited
5external paths actually executed
2paths where the compiler itself ran
5,419reference tasks screened for structure
17,836reference actions screened
1source withheld upstream, nothing imputed

What this audit does not claim

These flags are recorded in the evidence file itself and are fail-closed, so the matrix cannot quietly upgrade weaker evidence:

  • gated_source_metrics_imputed is false
  • screening_is_compiler_execution is false
  • screening_is_quality_evaluation is false
  • simulated_benchmarks_are_real_world_demos is false
  • task_only_zero_coverage_is_failure is false

What the numbers mean

Reference-plan screening

Reading a benchmark's own gold or reference action sequences to see whether any compilable read structure exists. No compiler runs and no model is called, so screening is never a quality result.

Candidate region

A contiguous run of read-like actions inside one task that could, in principle, be compiled. Counted per task as a candidate window.

Candidate family

A recurring call chain shared across tasks. Support counts how many tasks share that exact chain; the exact gate needs far more independent groups than any of these reach.

Barrier action

An action the compiler refuses to cross, because it writes, or its effect cannot be established as read-only.

Unknown-effect action

An action whose effect class could not be determined from the source. Treated as a barrier, never as a read.

Complete observed trace

A recorded task where every intermediate call result is retained, which is what a post-trace compiler needs to reconstruct provenance.

Independent groups

Calibration units that the exact gate counts. The configured gate requires 92 zero-violation groups before it will admit anything.

Gate outcome

The compiler's decision for a family. RETIRE means the evidence was insufficient and nothing is deployed, which is the default.

The eight supplementary benchmarks primarily test whether an agent can choose and execute actions or produce a final answer. That is a different question from the one asked here, which is whether an already-valid trace carries enough evidence to compile and admit a guarded program.

Reading the evidence status

measured

Compiler executed on this benchmark; a gate decision was produced.

screened

Reference plans were screened for compilable structure. No compiler ran.

gated

Upstream access denied. No task or compiler metric is imputed.

preflight

Real records validated and frozen, but the study has not been run.