1 lightning.finite_horizon

lightning.finite_horizon

Finite-horizon QH admission, optimization, certification, and publication.

1.1 Package guide

experiment.py is the composition and lifecycle owner. QhExperiment admits stage populations, constructs the data module, Lightning module and trainer, and coordinates fitting, checkpoint selection, evaluation and publication. inference_bundle.py implements immutable manifest/artifact checks, scorer reconstruction and bundle writing. The experiment retains transaction ordering and class dispatch; the bundle implementation does not construct training data. diagnostics owns research evidence: checkpoints.py authenticates receipt/checkpoint roles before loading, evaluation.py owns original-label H1 controls and supported fitted-target residuals, history.py admits telemetry, and population.py preserves identities and the causal TRAIN root view. selection.py owns closed selection rules and saved-epoch ranking. The experiment retains lifecycle ordering, loader/device dispatch and its late source-fingerprint callback. Complete-scene H1 selection and filtered-support canaries retain distinct population contracts. Shared training machinery remains in training. Campaign protocols, budgets and progression decisions stay outside the experiment.

All QH datasets use one actor-state contract for representation compatibility and root-qualified stage provenance for exact population identity. The checkpoint actor identity hashes both the representation contract and complete TRAIN provenance. Stage representations must agree; source manifests, rows, scenes and stored rollout identities remain checked separately.

Current loading rejects retired actor-contract and receipt formats. Original artifacts and historical evidence remain unchanged. The supported current path is a separately admitted fit with its own receipt and checkpoint; there is no transparent conversion or cross-population warm start. Composition does not relax supervision profiles or dense-valid requirements.

The artifact-dependent CPU completion proof is invoked explicitly with ARIA_QH_VERIFICATION_MANIFEST and tests/lightning/qh_corpus_checkpoint_smoke.py. Its input records owner-resolved checkpoint, receipt, configuration, protocol, corpus and source identities. Missing admissible multi-root artifacts leave this gate unmet; ordinary fixture tests and successful inspection do not establish checkpoint training admission.

nbv-qh (or python -m aria_nbv.lightning.finite_horizon.cli) provides two thin commands:

nbv-qh diagnostic-fit --config diagnostic.toml --check
nbv-qh diagnostic-fit --config diagnostic.toml --seed 41 --output-dir new-fit
nbv-qh verify-bundle --bundle published-bundle --manifest-sha256 EXTERNAL_DIGEST

Diagnostic TOML contains [experiment] (a QhExperimentConfig) and [request] (a QhDiagnosticFitRequest), with nested scorer/module/trainer and dataset configuration. An omitted optional request test population means no test stage. CLI seed/output overrides take precedence over TOML. --check validates typed configuration and prints resolved inputs without constructing a dataset, scorer, trainer or fit. It does not establish source admission. diagnostic-fit dispatches through QhExperiment.fit_diagnostic; campaign budgets are the caller’s concern. verify-bundle checks the externally supplied manifest identity, all payloads and runtime/source contracts, then reconstructs an evaluation-mode scorer on CPU by default. It does not establish policy effectiveness or resume training.

1.2 Evidence ownership

studies owns frozen architecture protocols, first-model Q1 and continuing-QH receipt admission, and post-hoc reductions. Generic diagnostic history supplies telemetry and common protocol bindings; each reader retains its own execution/disposition rules. reporting consumes admitted products for presentation. Campaign runners own frozen budgets and progression decisions; the experiment coordinates lifecycle transactions.

1.3 Fit and inference lifecycle

Finite-horizon fitting uses the canonical QhExperiment lifecycle through Python or the thin nbv-qh CLI described above. The normal lifecycle is:

Code

flowchart LR
  V[("VIN actor store")] --> D["QhDatasetConfig"]
  R[("Train · validation · test rollout stores")] --> D
  D --> M["QhDataModule admission"]
  M --> L["QhLightningModule"]
  A["Authoritative action_mask"] --> L
  L --> E["QhExperiment.fit"]
  E --> B[("Immutable bundle and receipts")]
  B --> C["Held-out and exact-Q2 certification"]
  B --> I["load_for_inference"]
  I --> O["Online QH adapter"]
  A --> O
  O --> S["Hard-valid CandidateScores"]

  classDef input fill:#D5E8D4,stroke:#82B366,color:#17202A,stroke-width:1.5px,rx:0,ry:0;
  classDef output fill:#F8CECC,stroke:#B85450,color:#17202A,stroke-width:1.5px,rx:0,ry:0;
  classDef compute fill:#E1D5E7,stroke:#9673A6,color:#17202A,stroke-width:1.5px,rx:8,ry:8;
  classDef data fill:#F5F5F5,stroke:#9E9E9E,color:#17202A,stroke-width:1.2px,rx:0,ry:0;
  class A input;
  class D,M,L,E,C,I,O compute;
  class V,R,B data;
  class S output;

flowchart LR
  V[("VIN actor store")] --> D["QhDatasetConfig"]
  R[("Train · validation · test rollout stores")] --> D
  D --> M["QhDataModule admission"]
  M --> L["QhLightningModule"]
  A["Authoritative action_mask"] --> L
  L --> E["QhExperiment.fit"]
  E --> B[("Immutable bundle and receipts")]
  B --> C["Held-out and exact-Q2 certification"]
  B --> I["load_for_inference"]
  I --> O["Online QH adapter"]
  A --> O
  O --> S["Hard-valid CandidateScores"]

  classDef input fill:#D5E8D4,stroke:#82B366,color:#17202A,stroke-width:1.5px,rx:0,ry:0;
  classDef output fill:#F8CECC,stroke:#B85450,color:#17202A,stroke-width:1.5px,rx:0,ry:0;
  classDef compute fill:#E1D5E7,stroke:#9673A6,color:#17202A,stroke-width:1.5px,rx:8,ry:8;
  classDef data fill:#F5F5F5,stroke:#9E9E9E,color:#17202A,stroke-width:1.2px,rx:0,ry:0;
  class A input;
  class D,M,L,E,C,I,O compute;
  class V,R,B data;
  class S output;

from aria_nbv.lightning.finite_horizon.experiment import (
    QhExperimentConfig,
    QhFitRequest,
)
from aria_nbv.lightning.finite_horizon.selection import QhCheckpointSelectionSpec
from aria_nbv.lightning.finite_horizon.module import QhLightningModuleConfig
from aria_nbv.vin.models.target_finite_horizon import (
    TargetFiniteHorizonScorerConfig,
)

experiment = QhExperimentConfig(
    scorer=TargetFiniteHorizonScorerConfig(max_horizon=4),
    module=QhLightningModuleConfig(
        actor_state_contract_hash="bound-during-fit",
        learning_contract_hash="bound-during-fit",
    ),
).setup_target()

result = experiment.fit(
    QhFitRequest(
        train=train_dataset_config,
        validation=validation_dataset_config,
        test=test_dataset_config,
        warm_start_from=None,
        checkpoint_selection=QhCheckpointSelectionSpec(),
        seed=23,
        output_bundle_dir=bundle_dir,
    )
)
runtime = experiment.load_for_inference(result.bundle, device="cuda")

The train, validation, and test configs must identify non-empty, scene-disjoint populations with compatible learning semantics. The output directory must not already exist. Fit publishes scorer weights, a manifest, and hashed training/checkpoint-selection receipts; it does not publish optimizer state as an inference dependency.

For a bounded one-epoch smoke, set the nested TrainerFactoryConfig to one epoch and one train/validation batch. The executable example is maintained in tests/lightning/test_qh_experiment.py::test_qh_fit_publishes_new_bundle_and_hashed_receipts.

1.3.1 Privileged diagnostic fitting

QhExperiment.fit_diagnostic(QhDiagnosticFitRequest(...)) runs the same data admission, Lightning objective, optimizer, validation, and checkpoint selection as fit. It writes a selected full Lightning checkpoint and a hashed diagnostic-receipt.json containing the concrete scorer/module/trainer configuration, population and scene identities, completed epochs and updates, and validation loss. It never creates a deployment manifest. Existing CF0 bundle publication rules remain in force.

from aria_nbv.lightning.finite_horizon.experiment import QhDiagnosticFitRequest

result = experiment.fit_diagnostic(
    QhDiagnosticFitRequest(
        train=train_dataset_config,
        validation=validation_dataset_config,
        test=test_dataset_config,
        checkpoint_selection=QhCheckpointSelectionSpec(),
        seed=41,
        output_dir=diagnostic_dir,
    )
)
module = experiment.load_diagnostic_checkpoint(result, device="cuda")

The loader verifies checkpoint and receipt digests before strict local reload; it returns an evaluation module, not an online deployment adapter or a resumed trainer. Only load trusted local checkpoints. Test populations are admitted for split checks but are not evaluated by fitting.

For a complete one-epoch commissioning run, set max_epochs=1 and both batch limits to 1.0 (the whole declared population). Select the GPU and mixed precision and set enable_validation=True through TrainerFactoryConfig. Its explicit W&B setting is preserved; use the existing WandbConfig(offline=True) for local logging. Report the actual chain counts and optimizer updates alongside epoch count, elapsed time, and peak allocated GPU memory. A tiny real-data commissioning population is a lifecycle check, not a benchmark.

The opt-in recipe at .configs/diagnostics/volumetric-vin.toml parses with QhExperimentConfig.from_toml(path). Supply existing scene-disjoint QhDatasetConfig populations to the request above; their provenance must also agree with the immutable actor-store splits. The recipe contains no machine-specific data paths and does not change campaign defaults.

1.4 Mask and Gradient Ownership

QhLightningModule.forward() exposes the scorer’s raw, action-mask-independent QhScoreOutput. Training then applies separate support contracts:

candidate_mask identifies materialized rows;
action_mask defines legal backup and policy support;
q_label_mask identifies factual one-step label support;
fitted-Q admission selects realized horizon targets;
step and padding masks exclude unrealized storage rows.

Feasibility learns from labelled valid and invalid materialized rows. Q loss and bootstrap participation remain zero outside Q-label and hard-action support. Zero-filled padding or invalid rows can therefore never win a masked selection, including when all valid Q values are negative.

1.5 Objective and Decoders

The fitted-Q objective uses Huber loss, online argmax/target evaluation, exact h -> h - 1 recursion, and hard-valid next-action support. Its implementation is framework-independent: objective.py receives online and target scorers and returns the complete training transaction: differentiable total loss, the utility-only objective, detached feasibility/ranking statistics, and scorer outputs already needed for logging. Feasibility BCE and ranking auxiliaries contribute to training; validation and checkpoint selection retain the utility loss alone. Candidate-wise versus state-equal feasibility reductions remain profile-specific. module.py performs the zero-support preflight before scorer queries, adapts the transaction to Lightning, and owns optimizer stepping, target synchronization, lifecycle admission, and checkpoint buffers. The scorer may use:

direct continuous regression, the canonical decoder; or
CORAL with a manifest-bound support and calibration identity.

Decoder selection belongs to TargetFiniteHorizonScorerConfig; Lightning dispatches the corresponding loss without changing mask semantics.

1.6 Bundle and Online Use

QhExperiment.load_for_inference() verifies the manifest, configuration, weights, trained horizons, representation semantics, calibration, geometry, and actor/learning identities. The online adapter in aria_nbv.oracle.pipelines.online_qh rejects mismatched contexts and converts only hard-valid conditional values into CandidateScores.

Learned feasibility is auxiliary in the deployed core. It does not replace the authoritative analytic/observed action mask.

Shared optimizer/gradient observations are documented in training. QH output diagnostics and completed-update accounting remain task-owned.

QH output diagnostics are independently configurable with module fields output_diagnostics_level (off, summary, detailed), output_diagnostics_every_n_updates, and output_diagnostics_max_items. Summary observes finite supported Q1 predictions, targets, absolute errors, validity logits and teacher fractions. Detailed adds capped scalar candidate samples. These are rank-local, candidate-weighted diagnostics, not the primary state-normalized ranking metrics. They reuse existing outputs without extra scorer calls or retained graphs and are disabled by default. The commissioning joint Q1/validity recipe sets a positive BCE weight of 1.0 as a pilot control. The pilot keeps float32 matrix multiplications at highest precision: TF32 rounding can change chart support for large world translations. Learned layers still use mixed precision. CUDA volume-sampling backward requires the recipe’s deterministic = false; this is a backend limit, not a reproducibility claim.

These metrics use the existing Lightning logger. The commissioning TOML enables W&B with offline = true; the diagnostic-cycle command builds its own trainer with use_wandb = false, which records no W&B run. The commissioning recipe uses an explicit fixed-rate AdamW pilot (learning_rate = 1e-3, weight_decay = 1e-3, scheduler disabled). Compare a lower fixed rate such as 1e-4 on the same admitted rows before treating either rate as chosen by validation. The historical OneCycle default instead initializes at 4e-6, peaks at 1e-4, and approaches 4e-10; declaring AdamW’s base LR alone does not control that trajectory.

The active thesis explanation of QH eligibility, masks, and finite-horizon learning is in docs/typst/thesis/sections/05-experimental-design/05-02-learning-objective-and-replay-evidence.typ. Detailed tensor shapes and failure contracts remain beside the implementing classes rather than being duplicated here.

1.7 Verification

cd aria_nbv
uv run ruff check aria_nbv/lightning
uv run pytest tests/data_handling/test_public_api_contract.py
uv run pytest tests/lightning/test_qh_datamodule.py
uv run pytest tests/lightning/test_qh_module.py
uv run pytest tests/lightning/test_qh_experiment.py
uv run pytest tests/lightning/test_qh_q2_certification.py
uv run pytest tests/lightning/test_grad_norm_logging.py

Use tests/lightning/test_qh_fast_dev_run.py for the smallest complete Lightning transaction.

1.8 Retained development-screen readback

finite_horizon.studies.development_screen verifies a frozen historical native-H1 screen: 24 planned source scenes, four held-out folds, scene-equal paired learner/constant regret, and the frozen 0.005 practical and improvement margins. It preserves unavailable acquisition intentions and includes measured-zero or noninformative usable scenes in the denominator. The constant’s frozen lowest-candidate-index tie rule and post-hoc uniform-tie sensitivity are separate readouts; the latter cannot change the frozen decision.

ScreenInput and ScreenResult bind the original assignment and terminal receipt bytes. read_development_screen reads available retained artifacts without reopening datasets or Git. seal_development_screen copies those artifacts and a verified captured aggregate into an exclusive portable bundle, returning its manifest SHA-256. Keep that hash independently. The portable reader requires all 22 intended assignment/admission/fit/checkpoint/evaluation and aggregate files; checkpoint bytes are hashed without loading a model. It rejects changed source or fit lineage, missing/duplicate paired support, nonfinite values, symlinks/traversal, and any inconsistent complete aggregate even after internal hashes are rebound.

from aria_nbv.lightning.finite_horizon.studies.development_screen import (
    read_portable_development_screen,
)

readback = read_portable_development_screen(
    manifest_path,
    expected_manifest_sha256=independently_retained_manifest_hash,
)

For command-line cold readback, run python -m aria_nbv.lightning.finite_horizon.studies.development_screen --manifest ... --expected-manifest-sha256 .... Original stores, historical Git, model loading and fitting are not required. This verifies captured admission consistency; it does not rerun canonical dataset admission. Portable output identifies historical_native_h1_v1, the captured source revision and receipt schemas, and the current native_horizon1_state_mean_v2 commissioning readout separately. Historical losses are preserved; current objective and commissioning owners remain authoritative for new numerical results.

finite_horizon.studies.development_ranking recomputes float32 label-separated pair ordering after paired screen validation and reports scene-equal accuracy with physical-scene bootstrap intervals. Pair/state counts within a scene are support, not independent replicates. Ranking diagnostics do not revise the frozen practical-regret decision or prove absence of learnability.

Fresh screen admission/fitting is not exposed by this reader package. A future runner requires current objective/readout binding, clean exact source and an outer process watchdog; the historical SIGALRM adapter does not qualify.

1.9 Resumable research fits

Diagnostic and TRAIN-canary fit requests accept an active_checkpoint_dir outside their fresh immutable output directory. The resume owner saves full checkpoints only after complete epochs. SIGINT/SIGTERM requests a pause after the current epoch; its interruption receipt identifies the exact checkpoint path, SHA-256, execution identity and successful-update count. SIGKILL recovers the latest completed epoch. A partial epoch stopped by max_steps does not become a resumable checkpoint.

Construct QhDiagnosticResumeRef.from_checkpoint(path) from a trusted local checkpoint and pass it as resume_from with a new output directory. The fit checks source, backend, configuration and reopened dataset identities before restoring online/target weights, optimizer state, RNG and continuing-query coverage. Exact recovery currently requires zero loader workers and uses an explicit epoch-deterministic sample order. Historical fits retain their original loader policy. A weights-only warm start and a full-state resume are different operations.

The Q1/QH research launchers expose --execution-profile mps-fp32, --active-checkpoint-dir and --resume-checkpoint. Online W&B continuation also requires the original --wandb-run-id, --wandb-resume and a fresh --attempt-id, plus --cold-initialization-lineage pointing to the original attempt’s untrained initialization artifact. Cloud identity alone does not restore training. See the artifact transfer workflow for immutable checkpoint snapshots and exact root/rollout dependencies.

# lightning.finite_horizon { #aria_nbv.lightning.finite_horizon } `lightning.finite_horizon` Finite-horizon QH admission, optimization, certification, and publication.  ## Package guide [experiment.py](https://github.com/JanDuchscherer104/ARIA-NBV/blob/main/aria_nbv/aria_nbv/lightning/finite_horizon/experiment.py) is the composition and lifecycle owner. `QhExperiment` admits stage populations, constructs the data module, Lightning module and trainer, and coordinates fitting, checkpoint selection, evaluation and publication. [inference_bundle.py](https://github.com/JanDuchscherer104/ARIA-NBV/blob/main/aria_nbv/aria_nbv/lightning/finite_horizon/inference_bundle.py) implements immutable manifest/artifact checks, scorer reconstruction and bundle writing. The experiment retains transaction ordering and class dispatch; the bundle implementation does not construct training data. [diagnostics](lightning.finite_horizon.diagnostics.qmd) owns research evidence: `checkpoints.py` authenticates receipt/checkpoint roles before loading, `evaluation.py` owns original-label H1 controls and supported fitted-target residuals, `history.py` admits telemetry, and `population.py` preserves identities and the causal TRAIN root view. [selection.py](https://github.com/JanDuchscherer104/ARIA-NBV/blob/main/aria_nbv/aria_nbv/lightning/finite_horizon/selection.py) owns closed selection rules and saved-epoch ranking. The experiment retains lifecycle ordering, loader/device dispatch and its late source-fingerprint callback. Complete-scene H1 selection and filtered-support canaries retain distinct population contracts. Shared training machinery remains in [training](lightning.training.qmd). Campaign protocols, budgets and progression decisions stay outside the experiment. All QH datasets use one actor-state contract for representation compatibility and root-qualified stage provenance for exact population identity. The checkpoint actor identity hashes both the representation contract and complete TRAIN provenance. Stage representations must agree; source manifests, rows, scenes and stored rollout identities remain checked separately. Current loading rejects retired actor-contract and receipt formats. Original artifacts and historical evidence remain unchanged. The supported current path is a separately admitted fit with its own receipt and checkpoint; there is no transparent conversion or cross-population warm start. Composition does not relax supervision profiles or dense-valid requirements. The artifact-dependent CPU completion proof is invoked explicitly with `ARIA_QH_VERIFICATION_MANIFEST` and `tests/lightning/qh_corpus_checkpoint_smoke.py`. Its input records owner-resolved checkpoint, receipt, configuration, protocol, corpus and source identities. Missing admissible multi-root artifacts leave this gate unmet; ordinary fixture tests and successful inspection do not establish checkpoint training admission. `nbv-qh` (or `python -m aria_nbv.lightning.finite_horizon.cli`) provides two thin commands: ```sh nbv-qh diagnostic-fit --config diagnostic.toml --check nbv-qh diagnostic-fit --config diagnostic.toml --seed 41 --output-dir new-fit nbv-qh verify-bundle --bundle published-bundle --manifest-sha256 EXTERNAL_DIGEST ``` Diagnostic TOML contains `[experiment]` (a `QhExperimentConfig`) and `[request]` (a `QhDiagnosticFitRequest`), with nested scorer/module/trainer and dataset configuration. An omitted optional request test population means no test stage. CLI seed/output overrides take precedence over TOML. `--check` validates typed configuration and prints resolved inputs without constructing a dataset, scorer, trainer or fit. It does not establish source admission. `diagnostic-fit` dispatches through `QhExperiment.fit_diagnostic`; campaign budgets are the caller's concern. `verify-bundle` checks the externally supplied manifest identity, all payloads and runtime/source contracts, then reconstructs an evaluation-mode scorer on CPU by default. It does not establish policy effectiveness or resume training. ## Evidence ownership [studies](lightning.finite_horizon.studies.qmd) owns frozen architecture protocols, first-model Q1 and continuing-QH receipt admission, and post-hoc reductions. Generic [diagnostic history](https://github.com/JanDuchscherer104/ARIA-NBV/blob/main/aria_nbv/aria_nbv/lightning/finite_horizon/diagnostics/history.py) supplies telemetry and common protocol bindings; each reader retains its own execution/disposition rules. [reporting](reporting.qmd) consumes admitted products for presentation. Campaign runners own frozen budgets and progression decisions; the experiment coordinates lifecycle transactions. ## Fit and inference lifecycle Finite-horizon fitting uses the canonical `QhExperiment` lifecycle through Python or the thin `nbv-qh` CLI described above. The normal lifecycle is: ```{mermaid} flowchart LR V[("VIN actor store")] --> D["QhDatasetConfig"] R[("Train · validation · test rollout stores")] --> D D --> M["QhDataModule admission"] M --> L["QhLightningModule"] A["Authoritative action_mask"] --> L L --> E["QhExperiment.fit"] E --> B[("Immutable bundle and receipts")] B --> C["Held-out and exact-Q2 certification"] B --> I["load_for_inference"] I --> O["Online QH adapter"] A --> O O --> S["Hard-valid CandidateScores"] classDef input fill:#D5E8D4,stroke:#82B366,color:#17202A,stroke-width:1.5px,rx:0,ry:0; classDef output fill:#F8CECC,stroke:#B85450,color:#17202A,stroke-width:1.5px,rx:0,ry:0; classDef compute fill:#E1D5E7,stroke:#9673A6,color:#17202A,stroke-width:1.5px,rx:8,ry:8; classDef data fill:#F5F5F5,stroke:#9E9E9E,color:#17202A,stroke-width:1.2px,rx:0,ry:0; class A input; class D,M,L,E,C,I,O compute; class V,R,B data; class S output; ``` ```python from aria_nbv.lightning.finite_horizon.experiment import ( QhExperimentConfig, QhFitRequest, ) from aria_nbv.lightning.finite_horizon.selection import QhCheckpointSelectionSpec from aria_nbv.lightning.finite_horizon.module import QhLightningModuleConfig from aria_nbv.vin.models.target_finite_horizon import ( TargetFiniteHorizonScorerConfig, ) experiment = QhExperimentConfig( scorer=TargetFiniteHorizonScorerConfig(max_horizon=4), module=QhLightningModuleConfig( actor_state_contract_hash="bound-during-fit", learning_contract_hash="bound-during-fit", ), ).setup_target() result = experiment.fit( QhFitRequest( train=train_dataset_config, validation=validation_dataset_config, test=test_dataset_config, warm_start_from=None, checkpoint_selection=QhCheckpointSelectionSpec(), seed=23, output_bundle_dir=bundle_dir, ) ) runtime = experiment.load_for_inference(result.bundle, device="cuda") ``` The train, validation, and test configs must identify non-empty, scene-disjoint populations with compatible learning semantics. The output directory must not already exist. Fit publishes scorer weights, a manifest, and hashed training/checkpoint-selection receipts; it does not publish optimizer state as an inference dependency. For a bounded one-epoch smoke, set the nested `TrainerFactoryConfig` to one epoch and one train/validation batch. The executable example is maintained in `tests/lightning/test_qh_experiment.py::test_qh_fit_publishes_new_bundle_and_hashed_receipts`. ### Privileged diagnostic fitting `QhExperiment.fit_diagnostic(QhDiagnosticFitRequest(...))` runs the same data admission, Lightning objective, optimizer, validation, and checkpoint selection as `fit`. It writes a selected full Lightning checkpoint and a hashed `diagnostic-receipt.json` containing the concrete scorer/module/trainer configuration, population and scene identities, completed epochs and updates, and validation loss. It never creates a deployment manifest. Existing CF0 bundle publication rules remain in force. ```python from aria_nbv.lightning.finite_horizon.experiment import QhDiagnosticFitRequest result = experiment.fit_diagnostic( QhDiagnosticFitRequest( train=train_dataset_config, validation=validation_dataset_config, test=test_dataset_config, checkpoint_selection=QhCheckpointSelectionSpec(), seed=41, output_dir=diagnostic_dir, ) ) module = experiment.load_diagnostic_checkpoint(result, device="cuda") ``` The loader verifies checkpoint and receipt digests before strict local reload; it returns an evaluation module, not an online deployment adapter or a resumed trainer. Only load trusted local checkpoints. Test populations are admitted for split checks but are not evaluated by fitting. For a complete one-epoch commissioning run, set `max_epochs=1` and both batch limits to `1.0` (the whole declared population). Select the GPU and mixed precision and set `enable_validation=True` through `TrainerFactoryConfig`. Its explicit W&B setting is preserved; use the existing `WandbConfig(offline=True)` for local logging. Report the actual chain counts and optimizer updates alongside epoch count, elapsed time, and peak allocated GPU memory. A tiny real-data commissioning population is a lifecycle check, not a benchmark. The opt-in recipe at `.configs/diagnostics/volumetric-vin.toml` parses with `QhExperimentConfig.from_toml(path)`. Supply existing scene-disjoint `QhDatasetConfig` populations to the request above; their provenance must also agree with the immutable actor-store splits. The recipe contains no machine-specific data paths and does not change campaign defaults. ## Mask and Gradient Ownership `QhLightningModule.forward()` exposes the scorer's raw, action-mask-independent `QhScoreOutput`. Training then applies separate support contracts: - `candidate_mask` identifies materialized rows; - `action_mask` defines legal backup and policy support; - `q_label_mask` identifies factual one-step label support; - fitted-Q admission selects realized horizon targets; - step and padding masks exclude unrealized storage rows. Feasibility learns from labelled valid and invalid materialized rows. Q loss and bootstrap participation remain zero outside Q-label and hard-action support. Zero-filled padding or invalid rows can therefore never win a masked selection, including when all valid Q values are negative. ## Objective and Decoders The fitted-Q objective uses Huber loss, online argmax/target evaluation, exact `h -> h - 1` recursion, and hard-valid next-action support. Its implementation is framework-independent: `objective.py` receives online and target scorers and returns the complete training transaction: differentiable total loss, the utility-only objective, detached feasibility/ranking statistics, and scorer outputs already needed for logging. Feasibility BCE and ranking auxiliaries contribute to training; validation and checkpoint selection retain the utility loss alone. Candidate-wise versus state-equal feasibility reductions remain profile-specific. `module.py` performs the zero-support preflight before scorer queries, adapts the transaction to Lightning, and owns optimizer stepping, target synchronization, lifecycle admission, and checkpoint buffers. The scorer may use: - direct continuous regression, the canonical decoder; or - CORAL with a manifest-bound support and calibration identity. Decoder selection belongs to `TargetFiniteHorizonScorerConfig`; Lightning dispatches the corresponding loss without changing mask semantics. ## Bundle and Online Use `QhExperiment.load_for_inference()` verifies the manifest, configuration, weights, trained horizons, representation semantics, calibration, geometry, and actor/learning identities. The online adapter in `aria_nbv.oracle.pipelines.online_qh` rejects mismatched contexts and converts only hard-valid conditional values into `CandidateScores`. Learned feasibility is auxiliary in the deployed core. It does not replace the authoritative analytic/observed action mask. Shared optimizer/gradient observations are documented in [training](lightning.training.qmd#optimizer-and-gradient-observation). QH output diagnostics and completed-update accounting remain task-owned. QH output diagnostics are independently configurable with module fields `output_diagnostics_level` (`off`, `summary`, `detailed`), `output_diagnostics_every_n_updates`, and `output_diagnostics_max_items`. Summary observes finite supported Q1 predictions, targets, absolute errors, validity logits and teacher fractions. Detailed adds capped scalar candidate samples. These are rank-local, candidate-weighted diagnostics, not the primary state-normalized ranking metrics. They reuse existing outputs without extra scorer calls or retained graphs and are disabled by default. The commissioning joint Q1/validity recipe sets a positive BCE weight of 1.0 as a pilot control. The pilot keeps float32 matrix multiplications at `highest` precision: TF32 rounding can change chart support for large world translations. Learned layers still use mixed precision. CUDA volume-sampling backward requires the recipe's `deterministic = false`; this is a backend limit, not a reproducibility claim. These metrics use the existing Lightning logger. The commissioning TOML enables W&B with `offline = true`; the diagnostic-cycle command builds its own trainer with `use_wandb = false`, which records no W&B run. The commissioning recipe uses an explicit fixed-rate AdamW pilot (`learning_rate = 1e-3`, `weight_decay = 1e-3`, scheduler disabled). Compare a lower fixed rate such as `1e-4` on the same admitted rows before treating either rate as chosen by validation. The historical OneCycle default instead initializes at `4e-6`, peaks at `1e-4`, and approaches `4e-10`; declaring AdamW's base LR alone does not control that trajectory. The active thesis explanation of QH eligibility, masks, and finite-horizon learning is in `docs/typst/thesis/sections/05-experimental-design/05-02-learning-objective-and-replay-evidence.typ`. Detailed tensor shapes and failure contracts remain beside the implementing classes rather than being duplicated here. ## Verification ```sh cd aria_nbv uv run ruff check aria_nbv/lightning uv run pytest tests/data_handling/test_public_api_contract.py uv run pytest tests/lightning/test_qh_datamodule.py uv run pytest tests/lightning/test_qh_module.py uv run pytest tests/lightning/test_qh_experiment.py uv run pytest tests/lightning/test_qh_q2_certification.py uv run pytest tests/lightning/test_grad_norm_logging.py ``` Use `tests/lightning/test_qh_fast_dev_run.py` for the smallest complete Lightning transaction. ## Retained development-screen readback `finite_horizon.studies.development_screen` verifies a frozen historical native-H1 screen: 24 planned source scenes, four held-out folds, scene-equal paired learner/constant regret, and the frozen 0.005 practical and improvement margins. It preserves unavailable acquisition intentions and includes measured-zero or noninformative usable scenes in the denominator. The constant's frozen lowest-candidate-index tie rule and post-hoc uniform-tie sensitivity are separate readouts; the latter cannot change the frozen decision. `ScreenInput` and `ScreenResult` bind the original assignment and terminal receipt bytes. `read_development_screen` reads available retained artifacts without reopening datasets or Git. `seal_development_screen` copies those artifacts and a verified captured aggregate into an exclusive portable bundle, returning its manifest SHA-256. Keep that hash independently. The portable reader requires all 22 intended assignment/admission/fit/checkpoint/evaluation and aggregate files; checkpoint bytes are hashed without loading a model. It rejects changed source or fit lineage, missing/duplicate paired support, nonfinite values, symlinks/traversal, and any inconsistent complete aggregate even after internal hashes are rebound. ```python from aria_nbv.lightning.finite_horizon.studies.development_screen import ( read_portable_development_screen, ) readback = read_portable_development_screen( manifest_path, expected_manifest_sha256=independently_retained_manifest_hash, ) ``` For command-line cold readback, run `python -m aria_nbv.lightning.finite_horizon.studies.development_screen --manifest ... --expected-manifest-sha256 ...`. Original stores, historical Git, model loading and fitting are not required. This verifies captured admission consistency; it does not rerun canonical dataset admission. Portable output identifies `historical_native_h1_v1`, the captured source revision and receipt schemas, and the current `native_horizon1_state_mean_v2` commissioning readout separately. Historical losses are preserved; current objective and commissioning owners remain authoritative for new numerical results. `finite_horizon.studies.development_ranking` recomputes float32 label-separated pair ordering after paired screen validation and reports scene-equal accuracy with physical-scene bootstrap intervals. Pair/state counts within a scene are support, not independent replicates. Ranking diagnostics do not revise the frozen practical-regret decision or prove absence of learnability. Fresh screen admission/fitting is not exposed by this reader package. A future runner requires current objective/readout binding, clean exact source and an outer process watchdog; the historical SIGALRM adapter does not qualify. ## Resumable research fits Diagnostic and TRAIN-canary fit requests accept an `active_checkpoint_dir` outside their fresh immutable output directory. The [resume owner](https://github.com/JanDuchscherer104/ARIA-NBV/blob/main/aria_nbv/aria_nbv/lightning/finite_horizon/resume.py) saves full checkpoints only after complete epochs. SIGINT/SIGTERM requests a pause after the current epoch; its interruption receipt identifies the exact checkpoint path, SHA-256, execution identity and successful-update count. SIGKILL recovers the latest completed epoch. A partial epoch stopped by `max_steps` does not become a resumable checkpoint. Construct `QhDiagnosticResumeRef.from_checkpoint(path)` from a trusted local checkpoint and pass it as `resume_from` with a new output directory. The fit checks source, backend, configuration and reopened dataset identities before restoring online/target weights, optimizer state, RNG and continuing-query coverage. Exact recovery currently requires zero loader workers and uses an explicit epoch-deterministic sample order. Historical fits retain their original loader policy. A weights-only warm start and a full-state resume are different operations. The Q1/QH research launchers expose `--execution-profile mps-fp32`, `--active-checkpoint-dir` and `--resume-checkpoint`. Online W&B continuation also requires the original `--wandb-run-id`, `--wandb-resume` and a fresh `--attempt-id`, plus `--cold-initialization-lineage` pointing to the original attempt's untrained initialization artifact. Cloud identity alone does not restore training. See the [artifact transfer workflow](https://github.com/JanDuchscherer104/ARIA-NBV/blob/main/experiments/2026-10-research/m5-benchmark/sync.md) for immutable checkpoint snapshots and exact root/rollout dependencies.