1 reporting.generation_throughput
reporting.generation_throughput
Evaluate the frozen, three-scene computational throughput assignment.
Oracle owns admission, tensor comparisons and the paired log-cost reducer. This owner records their verdict in SENPAI schema-v2; it adds no statistical gate or scientific computation. W&B publication remains a separate operation of aria_nbv.reporting.performance_evidence.
1.1 Functions
| Name | Description |
|---|---|
| evaluate_assignment | Write a new, immutable evaluator snapshot after canonical validation. |
| evaluator_fingerprint | Hash the supported evaluator and shared measurement reader bytes. |
| main | Run the real evaluator; return zero only for its retained-gain verdict. |
1.1.1 evaluate_assignment
reporting.generation_throughput.evaluate_assignment(
assignment_path,
*,
brief_path,
output_dir,
goal_slug,
)Write a new, immutable evaluator snapshot after canonical validation.
Counts describe the fixed 18-attempt engineering roster. The three scenes and two order-balanced production blocks are not a population estimate. Missing cost reductions remain absent metrics, never zero speedups. PASS requires the Oracle reader’s complete scientific, resource and frozen observed-variation verdict. No learning or corpus-admission claim follows.
1.1.2 evaluator_fingerprint
reporting.generation_throughput.evaluator_fingerprint()Hash the supported evaluator and shared measurement reader bytes.
Paths are package-relative to make this identity independent of checkout location. This fingerprint is frozen before candidate acquisition; the candidate’s single computation patch must leave these owners unchanged.
1.1.3 main
reporting.generation_throughput.main()Run the real evaluator; return zero only for its retained-gain verdict.