1 lightning.finite_horizon.module

lightning.finite_horizon.module

Scorer-independent fitted-Q optimization for finite-candidate chains.

The injected scorer maps QhActorTensors to QhScoreOutput with raw conditional values and feasibility logits. The objective module owns mask use, Double-Q targets, and horizon reduction. This adapter owns distributed admission, optimizer transactions, lifecycle metrics, and target sync.

1.1 Classes

Name	Description
QhLightningModule	Optimize selected finite-horizon transitions with Double-Q targets.
QhLightningModuleConfig	Configure fitted-Q optimization without constructing a scorer.

1.1.1 QhLightningModule

lightning.finite_horizon.module.QhLightningModule(config, *, scorer)

Optimize selected finite-horizon transitions with Double-Q targets.

The online scorer selects the supported successor action and the frozen target scorer evaluates it:

\[ a^*=\arg\max_{a\in\mathcal{A}_{action}\cap\mathcal{A}_{label}} Q_{online}(s',a),\qquad y=r+\gamma Q_{target}(s',a^*). \]

Manual optimization is restricted to the admitted-row transaction. All ranks first all-reduce the selected-row count. A globally empty batch is an exact no-op; otherwise every rank scores and steps once, with locally empty ranks contributing a zero connected to the online scorer graph.

1.1.1.1 Parameters

Name	Type	Description	Default
config	QhLightningModuleConfig	Optimizer, scheduler, regression-Huber, feasibility, and target-sync policy. A CORAL scorer supplies its ordinal loss metadata through :class:QhScoreOutput.	required
scorer	nn.Module	Required actor-only module returning :class:QhScoreOutput.	required

1.1.1.2 Attributes

Name	Description
optimizer_updates	Tensor["", int64] checkpointed count of real optimizer updates.
training_loss_sum	Tensor["", float32\|float64] detached local utility-loss sum this epoch.
training_row_count	Tensor["", int64] local admitted state–horizon contributions this epoch.
validation_loss_sum	Tensor["", float32\|float64] detached sum of active validation horizon means.
validation_row_count	Tensor["", int64] number of non-empty validation horizons.
test_loss_sum	Tensor["", float32\|float64] detached sum of active held-out horizon means.
test_row_count	Tensor["", int64] number of non-empty held-out horizons.

1.1.1.3 Methods

Name	Description
configure_resume_identity	Bind checkpoints from this module to one admitted fit identity.
on_save_checkpoint	Persist non-tensor fit state and process RNGs in full checkpoints.
on_load_checkpoint	Admit a compatible full-state checkpoint before Lightning restores it.
forward	Return raw online predictions with the actor’s exact batch shape.
train	Propagate mode to the online scorer while keeping the target in eval.
on_fit_start	Reject lifecycle settings incompatible with the exact transaction.
on_validation_start	Reject validation when the attached DataModule contract has drifted.
on_test_start	Reject testing when the attached DataModule contract has drifted.
on_predict_start	Reject any supported prediction lifecycle after DataModule drift.
training_step	Execute one globally admitted optimizer transaction or an exact no-op.
on_before_optimizer_step	Observe and clip gradients after Lightning precision has unscaled them.
on_train_epoch_start	Reset local loss and per-horizon support aggregates for a new epoch.
on_train_epoch_end	Make realized per-horizon support exact across distributed ranks.
validation_step	Accumulate exact single-device validation loss.
test_step	Accumulate exact single-device held-out loss.
on_validation_epoch_start	Reset validation numerators, denominators, and horizon counts.
on_validation_epoch_end	Finalize and log the equal-horizon validation loss.
on_test_epoch_start	Reset held-out numerators, denominators, and horizon counts.
on_test_epoch_end	Finalize and log the equal-horizon held-out loss.
compute_fitted_q_loss	Return the historical selected-diagonal diagnostic component.
compute_learning_objective	Return the executable dense-Q1 plus selected-recursion objective.
compute_feasibility_loss	Return auxiliary BCE over materialized valid and invalid rows.
compute_exact_q2_targets	Return dense-successor exact-Q2 targets and their factual support.
configure_optimizers	Construct AdamW over the online scorer and its optional scheduler.

1.1.1.3.1 configure_resume_identity

lightning.finite_horizon.module.QhLightningModule.configure_resume_identity(
    execution_identity_sha256,
)

Bind checkpoints from this module to one admitted fit identity.

1.1.1.3.2 on_save_checkpoint

lightning.finite_horizon.module.QhLightningModule.on_save_checkpoint(checkpoint)

Persist non-tensor fit state and process RNGs in full checkpoints.

1.1.1.3.3 on_load_checkpoint

lightning.finite_horizon.module.QhLightningModule.on_load_checkpoint(checkpoint)

Admit a compatible full-state checkpoint before Lightning restores it.

1.1.1.3.4 forward

lightning.finite_horizon.module.QhLightningModule.forward(
    actor,
    *,
    requested_horizon=None,
)

Return raw online predictions with the actor’s exact batch shape.

1.1.1.3.4.1 Parameters

Name	Type	Description	Default
actor	QhActorTensors	Actor-visible chain tensors whose action_mask defines the required Tensor["B S N", bool] output shape.	required

1.1.1.3.4.2 Returns

Name	Type	Description
	QhScoreOutput	Candidate-aligned conditional Q and feasibility logits. Policy
	QhScoreOutput	masks are intentionally not applied to this raw result.

1.1.1.3.5 train

lightning.finite_horizon.module.QhLightningModule.train(mode=True)

Propagate mode to the online scorer while keeping the target in eval.

1.1.1.3.6 on_fit_start

lightning.finite_horizon.module.QhLightningModule.on_fit_start()

Reject lifecycle settings incompatible with the exact transaction.

1.1.1.3.7 on_validation_start

lightning.finite_horizon.module.QhLightningModule.on_validation_start()

Reject validation when the attached DataModule contract has drifted.

1.1.1.3.8 on_test_start

lightning.finite_horizon.module.QhLightningModule.on_test_start()

Reject testing when the attached DataModule contract has drifted.

1.1.1.3.9 on_predict_start

lightning.finite_horizon.module.QhLightningModule.on_predict_start()

Reject any supported prediction lifecycle after DataModule drift.

1.1.1.3.10 training_step

lightning.finite_horizon.module.QhLightningModule.training_step(
    batch,
    batch_idx,
)

Execute one globally admitted optimizer transaction or an exact no-op.

1.1.1.3.11 on_before_optimizer_step

lightning.finite_horizon.module.QhLightningModule.on_before_optimizer_step(
    optimizer,
)

Observe and clip gradients after Lightning precision has unscaled them.

Lightning invokes this hook once between unscale and the raw optimizer step. No scaler method is called here. Optional fail-fast admission is independent of logging cadence, and precedes clipping so invalid gradients cannot be hidden by a diagnostic sampling decision.

1.1.1.3.12 on_train_epoch_start

lightning.finite_horizon.module.QhLightningModule.on_train_epoch_start()

Reset local loss and per-horizon support aggregates for a new epoch.

1.1.1.3.13 on_train_epoch_end

lightning.finite_horizon.module.QhLightningModule.on_train_epoch_end()

Make realized per-horizon support exact across distributed ranks.

1.1.1.3.14 validation_step

lightning.finite_horizon.module.QhLightningModule.validation_step(
    batch,
    batch_idx,
)

Accumulate exact single-device validation loss.

1.1.1.3.15 test_step

lightning.finite_horizon.module.QhLightningModule.test_step(batch, batch_idx)

Accumulate exact single-device held-out loss.

1.1.1.3.16 on_validation_epoch_start

lightning.finite_horizon.module.QhLightningModule.on_validation_epoch_start()

Reset validation numerators, denominators, and horizon counts.

1.1.1.3.17 on_validation_epoch_end

lightning.finite_horizon.module.QhLightningModule.on_validation_epoch_end()

Finalize and log the equal-horizon validation loss.

1.1.1.3.18 on_test_epoch_start

lightning.finite_horizon.module.QhLightningModule.on_test_epoch_start()

Reset held-out numerators, denominators, and horizon counts.

1.1.1.3.19 on_test_epoch_end

lightning.finite_horizon.module.QhLightningModule.on_test_epoch_end()

Finalize and log the equal-horizon held-out loss.

1.1.1.3.20 compute_fitted_q_loss

lightning.finite_horizon.module.QhLightningModule.compute_fitted_q_loss(batch)

Return the historical selected-diagonal diagnostic component.

This compatibility diagnostic includes selected h=1 rows. Actual optimization is owned by compute_learning_objective, which replaces selected h=1 with the complete dense candidate table and keeps selected recursion only for h>1.

1.1.1.3.21 compute_learning_objective

lightning.finite_horizon.module.QhLightningModule.compute_learning_objective(
    batch,
)

Return the executable dense-Q1 plus selected-recursion objective.

This pure helper performs the same actor-only scorer calls, target construction, candidate/state normalization, and equal-horizon aggregation used by training. It does not mutate optimizer or target network state and is therefore the narrow inspection seam for tests and experiment diagnostics.

1.1.1.3.22 compute_feasibility_loss

lightning.finite_horizon.module.QhLightningModule.compute_feasibility_loss(
    batch,
)

Return auxiliary BCE over materialized valid and invalid rows.

1.1.1.3.23 compute_exact_q2_targets

lightning.finite_horizon.module.QhLightningModule.compute_exact_q2_targets(
    batch,
)

Return dense-successor exact-Q2 targets and their factual support.

This diagnostic contains no learned successor value: the second-step maximum comes directly from persisted one-step candidate rewards. A row is exact only when every hard-valid successor action has a label; partial successor support cannot certify the maximum over the factual action set.

1.1.1.3.24 configure_optimizers

lightning.finite_horizon.module.QhLightningModule.configure_optimizers()

Construct AdamW over the online scorer and its optional scheduler.

1.1.2 QhLightningModuleConfig

lightning.finite_horizon.module.QhLightningModuleConfig()

Configure fitted-Q optimization without constructing a scorer.

1.1.2.1 Attributes

Name	Description
optimizer	AdamW settings applied only to the online scorer.
lr_scheduler	Optional stateful schedule stepped once after each real optimizer update.
use_lr_scheduler	Enable the configured schedule; false keeps AdamW at its configured learning rate.
gradient_clip_norm	Optional positive gradient norm bound applied after precision unscaling, before stepping.
output_diagnostics_level	Optional detached Q1 outputs; detailed adds capped per-candidate scalar samples.
output_diagnostics_every_n_updates	Observe Q1 output diagnostics at this completed-update cadence.
output_diagnostics_max_items	Maximum candidate samples emitted by detailed Q1 output diagnostics.
training_diagnostics	Default-off bounded training/gradient observations and optional fail-fast admission.
huber_delta	Positive transition point for regression and optional decoded-CORAL Huber.
loss_accumulation_dtype	Reduction/statistics dtype; matched MPS/FP32 cells explicitly select float32.
q1_coral_regression_loss_weight	Fixed native-Q Huber multiplier added to CORAL threshold BCE in H1-only fits.
objective_profile	Closed optimization support; both Q1 profiles omit recursive and target queries.
q1_pairwise_loss_weight	Native regression-loss-unit coefficient for dimensionless preference cost; historical H1 only.
q1_pairwise_temperature	Positive native-Q-unit scale of unequal-pair score differences; regression remains native.
q1_pairwise_gap_weighting	Multiply pair costs by native reward gap/temperature; pair/state denominators remain unchanged.
q1_listwise_loss_weight	Default-off coefficient for state-mean forward KL beside native Huber.
q1_listwise_temperature	Native-Q temperature shared by detached target and predicted softmax.
q1_bestgap_loss_weight	Default-off native-Huber auxiliary regressing exact-best versus rest gaps.
feasibility_loss_weight	Auxiliary binary-feasibility loss weight; zero preserves the A1 control.
action_mask_semantics	Named hard-validity teacher used by the auxiliary feasibility loss.
target_sync_interval	Hard target-copy cadence measured in completed optimizer updates.
max_horizon	Largest fitted horizon used to allocate exact horizon-wise sufficient statistics.
root_evl_profile	Exact root-EVL carrier the injected scorer accepts.
selected_observation_protocol	Exact selected-observation source the injected scorer accepts; privileged CF-GT is opt-in.
experiment_profile	Closed deployable role; CF+ is explicit and privileged.
privileged	Allow the CF+ upper-bound role; deployable modules must leave this false.
actor_state_contract_hash	Required exact fit-admission digest of actor representation and complete TRAIN provenance.
learning_contract_hash	Required stable hash of the complete effective rollout learning contract.
geometry_contract_hash	Expected selected-depth geometry hash; required only for CF+ admission.
target_type	Return the runtime module type used by config-as-factory setup.