1 reporting.scorer

reporting.scorer

Presentation of admitted scorer evidence; import its owning renderer directly.

1.1 Package guide

Lightning studies admit receipts, preserve scientific support and reduce physical scene observations. These renderers consume those admitted products; they do not fit models, select checkpoints, or promote failed runs into comparisons.

section.py projects evidence into the canonical report builder and bundle. first_model.py, continuing.py and posthoc.py retain standalone campaign outputs: detailed readiness tables, marked mission/report updates and exploratory plots. These differ from the canonical compact tables and Plotly scene contrasts; shared inputs do not make the products interchangeable.

The completed Mac Q1 campaign has a source-bound research adapter in mac_q1_research.py. build_mac_q1_synthesis admits the saved native report, pairs original supported candidates and reduces states within physical scenes. It creates immutable tables with fixed-weight, exploratory scene-bootstrap intervals. build_synthesis_figures consumes only those tables and returns Seaborn SVG/PNG assets plus canonical interactive ReportFigure counterparts. Neither operation loads checkpoints or performs model inference.

New native Mac reports use version two to retain successful final and selected checkpoint-update counts admitted against fit/evaluation receipts. Synthesis reads that saved product after original attempt directories are unavailable. Historical version-one products remain readable and byte-preserved; missing selected-update evidence stays unavailable. The explicit legacy_selected_update_receipts option reopens hash-bound old receipts when their original paths exist, without rewriting or upgrading the historical report.

From a configured repository Python environment:

python scripts/report_mac_q1_research.py /path/to/report-observer.json \
  --context /path/to/research-context.json \
  --output /path/to/research-synthesis

The authored context pins the exact native report bytes and the population support claimed by its prose. Equations resolve through the generated notation registry and render from canonical Typst facade bodies; terms use the generated glossary. The standalone bundle preserves the native parent, context, source references, shared ReportSnapshot, exact displayed assets and an integrity manifest. read_mac_q1_research_report verifies that complete bundle without reacquiring mutable sources. This is a standalone campaign adapter, rather than the generic recipe-driven report builder/exporter. Native observer reports and their source-quality receipts remain separate and unchanged.

Its current scientific comparison is specifically completed C0/C1 seed41; engineering results and planned cells cannot become scientific comparisons. New fits or changed cohorts require a newly reviewed context pin. Tests live in tests/reporting/test_mac_q1_synthesis.py, test_mac_q1_synthesis_figures.py and test_mac_q1_research.py.

Current report scripts dispatch here. Source kinds, CLI arguments, output paths and serialized snapshots retain their established contracts. Frozen historical producer copies remain provenance inputs and are never imported as runtime code.

See reporting, study admission, canonical report tests and detached products.

Canonical scorer sections leave historical products unchanged unless diagnostics are explicitly enabled. continuing_diagnostics adds sparse source-event training points and query-supported horizon residuals. continuing_cloud_diagnostics requires complete exact-run W&B acquisition and matching retained startup/failure controls; it preserves source-row coverage while collapsing only identical run/update diagnostic copies. Both are descriptive partial-fit products, without population intervals or failure-time state reconstruction. New enabled options participate in report identity; disabled options preserve sealed recipe hashes.

ScorerStudySourceConfig(kind="matched_qh") selects the prospective matched study’s explicit typed admission. It uses the same archive, report snapshot and bundle owners as historical studies, without relaxing their frozen protocols. The source identity retains fit (source_package_digest), evaluator (evaluator_source_package_digest) and executing analysis (analysis_source_sha256) package digests separately. The analysis digest conservatively covers the current package’s Python tree, including its imported reductions. It retains bootstrap_method, bootstrap_seed, bootstrap_draws, bootstrap_unit, reduction and conditioning from matched admission: 10,000 percentile-bootstrap draws seeded by 20261009 over physical scenes, following equal-state and within-scene equal-recipe reductions. Intervals condition on the exposed validation population and fixed weights/checkpoint; they do not measure training seed or checkpoint-selection uncertainty. These identities and settings survive cold bundle reads and participate in the snapshot digest. The section exposes cell and state dispositions, per-scene observations, per-seed paired differences and conditional intervals. Its single figure facets metrics and seeds with separate native scales; final is primary and selected is secondary. Immutable exports can be read without the original fit or checkpoint payloads. Figure-gallery registration and scientific admission remain separate publication gates. See test_matched_scorer_report.py.

The explicit interrupted_qh scorer source projects admitted failed-attempt cold and partial-weight diagnostics into existing report snapshots. Its archive prefix contains exactly cold.json and interrupted.json, authenticated by an external manifest pin. Scene contrast tables/figures show all 20 physical-scene observations and paired uncertainty, with recipe-level absolute metrics available separately. The two-row checkpoint metrics table retains equal-scene absolute score, teacher calibration/discrimination and analytic-uniform regret values for audit. Within-state AUROC retains per-state class support and displays all scene contrasts with an explicitly positive-favours-interrupted axis. Its intervals condition on weights and checkpoint selection, as the score contrasts do. Operational calibration shows supported physical-scene bins and separately named descriptive candidate-pooled curves; empty bins retain zero support and missing rates. Exported source identity retains evaluator and analysis digests, both receipt/checkpoint/module-state identities, original archive pin, bootstrap settings and bin edges. This new source kind adds no configuration defaults and leaves sealed historical report recipes unchanged.

# reporting.scorer { #aria_nbv.reporting.scorer } `reporting.scorer` Presentation of admitted scorer evidence; import its owning renderer directly.  ## Package guide Lightning studies admit receipts, preserve scientific support and reduce physical scene observations. These renderers consume those admitted products; they do not fit models, select checkpoints, or promote failed runs into comparisons. `section.py` projects evidence into the canonical report builder and bundle. `first_model.py`, `continuing.py` and `posthoc.py` retain standalone campaign outputs: detailed readiness tables, marked mission/report updates and exploratory plots. These differ from the canonical compact tables and Plotly scene contrasts; shared inputs do not make the products interchangeable. The completed Mac Q1 campaign has a source-bound research adapter in `mac_q1_research.py`. `build_mac_q1_synthesis` admits the saved native report, pairs original supported candidates and reduces states within physical scenes. It creates immutable tables with fixed-weight, exploratory scene-bootstrap intervals. `build_synthesis_figures` consumes only those tables and returns Seaborn SVG/PNG assets plus canonical interactive `ReportFigure` counterparts. Neither operation loads checkpoints or performs model inference. New native Mac reports use version two to retain successful final and selected checkpoint-update counts admitted against fit/evaluation receipts. Synthesis reads that saved product after original attempt directories are unavailable. Historical version-one products remain readable and byte-preserved; missing selected-update evidence stays unavailable. The explicit legacy_selected_update_receipts option reopens hash-bound old receipts when their original paths exist, without rewriting or upgrading the historical report. From a configured repository Python environment: ```sh python scripts/report_mac_q1_research.py /path/to/report-observer.json \ --context /path/to/research-context.json \ --output /path/to/research-synthesis ``` The authored context pins the exact native report bytes and the population support claimed by its prose. Equations resolve through the generated notation registry and render from canonical Typst facade bodies; terms use the generated glossary. The standalone bundle preserves the native parent, context, source references, shared `ReportSnapshot`, exact displayed assets and an integrity manifest. `read_mac_q1_research_report` verifies that complete bundle without reacquiring mutable sources. This is a standalone campaign adapter, rather than the generic recipe-driven report builder/exporter. Native observer reports and their source-quality receipts remain separate and unchanged. Its current scientific comparison is specifically completed C0/C1 seed41; engineering results and planned cells cannot become scientific comparisons. New fits or changed cohorts require a newly reviewed context pin. Tests live in `tests/reporting/test_mac_q1_synthesis.py`, `test_mac_q1_synthesis_figures.py` and `test_mac_q1_research.py`. Current report scripts dispatch here. Source kinds, CLI arguments, output paths and serialized snapshots retain their established contracts. Frozen historical producer copies remain provenance inputs and are never imported as runtime code. See [reporting](reporting.qmd), [study admission](lightning.finite_horizon.studies.qmd), [canonical report tests](https://github.com/JanDuchscherer104/ARIA-NBV/blob/main/aria_nbv/tests/reporting/test_scorer_study_report.py) and [detached products](https://github.com/JanDuchscherer104/ARIA-NBV/blob/main/aria_nbv/tests/lightning/test_scorer_study_products.py). Canonical scorer sections leave historical products unchanged unless diagnostics are explicitly enabled. `continuing_diagnostics` adds sparse source-event training points and query-supported horizon residuals. `continuing_cloud_diagnostics` requires complete exact-run W&B acquisition and matching retained startup/failure controls; it preserves source-row coverage while collapsing only identical run/update diagnostic copies. Both are descriptive partial-fit products, without population intervals or failure-time state reconstruction. New enabled options participate in report identity; disabled options preserve sealed recipe hashes. `ScorerStudySourceConfig(kind="matched_qh")` selects the prospective matched study's explicit typed admission. It uses the same archive, report snapshot and bundle owners as historical studies, without relaxing their frozen protocols. The source identity retains fit (`source_package_digest`), evaluator (`evaluator_source_package_digest`) and executing analysis (`analysis_source_sha256`) package digests separately. The analysis digest conservatively covers the current package's Python tree, including its imported reductions. It retains `bootstrap_method`, `bootstrap_seed`, `bootstrap_draws`, `bootstrap_unit`, `reduction` and `conditioning` from matched admission: 10,000 percentile-bootstrap draws seeded by 20261009 over physical scenes, following equal-state and within-scene equal-recipe reductions. Intervals condition on the exposed validation population and fixed weights/checkpoint; they do not measure training seed or checkpoint-selection uncertainty. These identities and settings survive cold bundle reads and participate in the snapshot digest. The section exposes cell and state dispositions, per-scene observations, per-seed paired differences and conditional intervals. Its single figure facets metrics and seeds with separate native scales; final is primary and selected is secondary. Immutable exports can be read without the original fit or checkpoint payloads. Figure-gallery registration and scientific admission remain separate publication gates. See [test_matched_scorer_report.py](https://github.com/JanDuchscherer104/ARIA-NBV/blob/main/aria_nbv/tests/reporting/test_matched_scorer_report.py). The explicit `interrupted_qh` scorer source projects admitted failed-attempt cold and partial-weight diagnostics into existing report snapshots. Its archive prefix contains exactly `cold.json` and `interrupted.json`, authenticated by an external manifest pin. Scene contrast tables/figures show all 20 physical-scene observations and paired uncertainty, with recipe-level absolute metrics available separately. The two-row checkpoint metrics table retains equal-scene absolute score, teacher calibration/discrimination and analytic-uniform regret values for audit. Within-state AUROC retains per-state class support and displays all scene contrasts with an explicitly positive-favours-interrupted axis. Its intervals condition on weights and checkpoint selection, as the score contrasts do. Operational calibration shows supported physical-scene bins and separately named descriptive candidate-pooled curves; empty bins retain zero support and missing rates. Exported source identity retains evaluator and analysis digests, both receipt/checkpoint/module-state identities, original archive pin, bootstrap settings and bin edges. This new source kind adds no configuration defaults and leaves sealed historical report recipes unchanged.