1 vin.models.scene_myopic

vin.models.scene_myopic

VIN v3 one-step RRI scorer with evidence-backed components.

This module owns the active VIN one-step baseline. It predicts per-row RRI scores for a finite candidate set and is used as a myopic scorer/control, not as the thesis finite-horizon value model. The most reliable signal in the current implementation comes from pose encoding, EVL voxel evidence, and semidense projection coverage, so v3 keeps a compact deterministic path with optional trajectory context behind a config flag:

1. Pose encoding (R6D + LFF): Candidate poses are expressed in the reference rig frame T_rig_ref_cam = T_world_rig_ref^{-1} * T_world_cam and encoded as translation plus rotation-6D with Learnable Fourier Features.

2. Scene field (fixed channels): The voxel field concatenates occ_pr, cent_pr, counts_norm, occ_input, free_input, and new_surface_prior. We normalize counts as counts_norm = log1p(n) / log1p(max(n)) and define unknown = 1 - counts_norm, new_surface_prior = unknown * occ_pr. This compact field was stable in sweep diagnostics and supports voxel-validity gating.

3. Global context (pose-conditioned attention): A pooled voxel grid is attended by pose embeddings, with LFF positional keys in the reference rig frame.

4. Semidense projection stats (VIN-NBV proxy): We project semidense points into each candidate view to compute coverage, empty fraction, visibility fraction, and depth moments. These features act as a lightweight proxy for frustum attention, are concatenated into the scorer input, and drive a tiny CNN over the projection grid for richer cues.

5. Voxel projection FiLM: Pooled voxel centers are projected into candidate views and summarized; this drives a light FiLM modulation of the global feature (kept as the only view-conditioned modulation).

6. Optional trajectory context (disabled by default): Snippet rig poses can be encoded and attended by candidate embeddings to provide motion context, mirroring the v2 path without forcing it on the baseline.

7. CORAL head: A shallow MLP plus CORAL ordinal head produces per-candidate RRI scores.

The forward contract is actor-visible: candidate poses, EVL features, and semidense observations are inputs; oracle RRI labels are training/evaluation targets. Target-conditioned rollout data may reuse the same architecture family, but matched GT targets and target crops must remain outside actor inputs.

Frame-consistency: Candidate generation applies rotate_yaw_cw90 (a local +Z roll) to poses for UI alignment. EVL backbone outputs do not use this convention. VinModelV3 therefore undoes this rotation before computing pose features. If apply_cw90_correction is enabled, callers must pre-correct p3d_cameras and set cw90_corrected=True.

NOTE: vin inputs are typically VinSnippetView with points_world shaped (N,4) or (N,5) containing (x, y, z, 1/sigma_d) with optional n_obs. This file enforces the required XYZ + reliability channel contract to avoid silent failure modes.

Candidate orientation uses R6D pose features. Accumulated target visibility, if added for target-conditioned value learning, should be represented as a separate directional memory over view directions, not folded into the pose encoding.

1.1 Classes

Name	Description
VinModelV3	VIN-Core head for one-step RRI prediction.
VinModelV3Config	Configuration for VinModelV3 (streamlined one-step VIN baseline).

1.1.1 VinModelV3

vin.models.scene_myopic.VinModelV3(config)

VIN-Core head for one-step RRI prediction.

VIN v3 focuses on pose encoding, compact voxel evidence, and semidense projection stats, while enforcing fail-fast contracts to avoid silent collapse. It ranks candidates for immediate RRI; bounded rollout values such as \(Q_H\) are separate thesis models trained on rollout replay.

Candidate-set symmetry is structural: learned blocks use row-wise maps, candidate queries over shared scene/trajectory tokens, and symmetric GroupNorm statistics across rows. There are no candidate-index embeddings or order-sensitive candidate operations. In deterministic evaluation, jointly permuting candidate poses and aligned camera rows therefore permutes every N_q output the same way; stochastic training dropout preserves only distributional equivariance. This is not graph-isomorphism invariance because the model constructs no candidate graph. Its reference-frame pose and positional encodings are coordinate-conditioned inductive biases and do not enforce SE(3) equivariance.

1.1.1.1 Attributes

Name	Description
pose_encoder_lff	Return the LFF sub-encoder when present (else None).

1.1.1.2 Methods

Name	Description
train	Set training mode and discard inference-only prepared scene state.
forward	Score an aligned candidate set with cached actor-visible evidence.
forward_with_debug	Run VIN v3 forward pass and return intermediate tensors.
init_bin_values	Initialize learnable bin representatives for CORAL expectation.
summarize_vin	Summarize VIN v3 inputs/outputs for a single oracle-labeled batch.

1.1.1.2.1 train

vin.models.scene_myopic.VinModelV3.train(mode=True)

Set training mode and discard inference-only prepared scene state.

1.1.1.2.2 forward

vin.models.scene_myopic.VinModelV3.forward(
    efm,
    candidate_poses_world_cam,
    reference_pose_world_rig,
    p3d_cameras,
    backbone_out=None,
)

Score an aligned candidate set with cached actor-visible evidence.

1.1.1.2.2.1 Parameters

Name	Type	Description	Default
efm	EfmSnippetView | VinSnippetView	EFM snippet view or VIN snippet view for the current snippet.	required
candidate_poses_world_cam	PoseTW	PoseTW["B N_q 12"] world-from-camera poses.	required
reference_pose_world_rig	PoseTW	PoseTW["B 12"] world-from-rig pose.	required
p3d_cameras	PerspectiveCameras	PyTorch3D cameras aligned with the flattened B * N_q candidate rows.	required
backbone_out	EvlBackboneOutput | None	Required cached actor-visible EVL evidence. None is accepted by the signature only to produce a fail-fast error.	None

1.1.1.2.2.2 Returns

Name	Type	Description
	VinPrediction	VinPrediction with Tensor["B N_q K-1", float32] CORAL logits,
	VinPrediction	Tensor["B N_q K", float32] probabilities, and aligned expected
	VinPrediction	scores and diagnostic validity fields.

1.1.1.2.3 forward_with_debug

vin.models.scene_myopic.VinModelV3.forward_with_debug(
    efm,
    candidate_poses_world_cam,
    reference_pose_world_rig,
    p3d_cameras,
    backbone_out=None,
)

Run VIN v3 forward pass and return intermediate tensors.

1.1.1.2.3.1 Parameters

Name	Type	Description	Default
efm	EfmSnippetView | VinSnippetView	EFM snippet view or VIN snippet view for the current snippet.	required
candidate_poses_world_cam	PoseTW	PoseTW["B N_q 12"] world-from-camera poses.	required
reference_pose_world_rig	PoseTW	PoseTW["B 12"] world-from-rig pose.	required
p3d_cameras	PerspectiveCameras	PyTorch3D cameras aligned with B * N_q rows.	required
backbone_out	EvlBackboneOutput | None	Required cached actor-visible EVL evidence.	None

1.1.1.2.3.2 Returns

Name	Type	Description
	tuple[VinPrediction, VinV3ForwardDiagnostics]	Prediction and B/N_q-aligned intermediate tensors.

1.1.1.2.4 init_bin_values

vin.models.scene_myopic.VinModelV3.init_bin_values(values, *, overwrite=False)

Initialize learnable bin representatives for CORAL expectation.

1.1.1.2.4.1 Parameters

Name	Type	Description	Default
values	Tensor	Tensor["K"] target bin representatives (e.g., bin means).	required
overwrite	bool	If True, overwrite existing bin values.	False

1.1.1.2.5 summarize_vin

vin.models.scene_myopic.VinModelV3.summarize_vin(
    batch,
    *,
    include_torchsummary=True,
    torchsummary_depth=3,
)

Summarize VIN v3 inputs/outputs for a single oracle-labeled batch.

This is intended for quick sanity checks when validating the streamlined baseline against sweep-derived expectations.

1.1.1.2.5.1 Parameters

Name	Type	Description	Default
batch	VinOracleBatch	Oracle-labeled batch to inspect.	required
include_torchsummary	bool	Whether to include a torchsummary block.	True
torchsummary_depth	int	Depth for torchsummary.	3

1.1.1.2.5.2 Returns

Name	Type	Description
	str	Human-readable summary string.

1.1.2 VinModelV3Config

vin.models.scene_myopic.VinModelV3Config()

Configuration for VinModelV3 (streamlined one-step VIN baseline).

1.1.2.1 Attributes

Name	Description
target_type	Factory target for BaseConfig.setup_target (config-as-factory).
backbone	Optional EVL config kept for caller-side materialization of voxel features.
pose_encoder	Pose encoder configuration (R6D + LFF; stable relative pose encoding).
pos_grid_encoder_lff	LFF encoder for XYZ voxel position keys used by global pooling.
head_hidden_dim	Hidden dimension for the scorer MLP (optuna favored compact heads).
head_num_layers	Number of MLP layers before the CORAL layer (best trials used 1).
head_dropout	Dropout probability in the MLP (sweep best used near-zero dropout).
num_classes	Number of ordinal bins (VIN-NBV uses 15 for sweep comparability).
coral_preinit_bias	Pre-initialize CORAL biases for faster, more stable ordinal learning.
field_dim	Channel dimension of the projected voxel field (compact by design).
field_gn_groups	Requested GroupNorm groups (clamped to a divisor of field_dim) for stability.
semidense_proj_grid_size	Grid size for semidense projection stats (higher for tiny-CNN cues).
semidense_proj_max_points	Maximum semidense points used for projection stats (sweep best used 4096).
semidense_cnn_enabled	Whether to encode a tiny 2D CNN over the semidense projection grid.
semidense_cnn_channels	Hidden channel width for the semidense projection CNN.
semidense_cnn_out_dim	Output feature dimension of the semidense projection CNN.
use_traj_encoder	Whether to encode snippet trajectories and append trajectory context.
traj_encoder	Optional trajectory encoder for snippet rig poses (R6D + LFF).
use_voxel_valid_frac_gate	Deprecated: voxel gate removed. Keep False (use FiLM only).
semidense_obs_count_min	Global minimum of semidense observation count n_obs (cache summary).
semidense_obs_count_max	Global maximum of semidense observation count n_obs (cache summary).
semidense_obs_count_p95	Global 95th percentile of semidense observation count n_obs (cache summary).
semidense_obs_count_mean	Global mean of semidense observation count n_obs (cache summary).
semidense_obs_count_std	Global standard deviation of semidense observation count n_obs (cache summary).
semidense_inv_dist_std_min	Global minimum of semidense inverse depth std 1/sigma_d (cache summary).
semidense_inv_dist_std_max	Global maximum of semidense inverse depth std 1/sigma_d (cache summary).
semidense_inv_dist_std_p95	Global 95th percentile of semidense inverse depth std 1/sigma_d (cache summary).
semidense_inv_dist_std_mean	Global mean of semidense inverse depth std 1/sigma_d (cache summary).
semidense_inv_dist_std_std	Global standard deviation of semidense inverse depth std 1/sigma_d (cache summary).
apply_cw90_correction	Undo rotate_yaw_cw90 on poses (requires CW90-corrected cameras).
global_pool_grid_size	Target grid size for pose-conditioned global pooling (best trials used ~5).
scene_field_channels	Ordered list of scene-field channels to include in the voxel field.