1 vin

vin

VIN one-step and finite-horizon scoring models.

The active public surface is VinModelV3: a target-agnostic one-step scorer for candidate RRI rows that uses EVL voxel evidence, candidate pose encodings, semidense projection features, and a CORAL ordinal head. It is the implemented seminar substrate and the myopic control for the thesis target-conditioned rollout work. The production finite-horizon scorer remains leaf-owned by aria_nbv.vin.models.target_finite_horizon; its modular history, scene, state-fusion, and value decoders live under aria_nbv.vin.modules, and aria_nbv.vin.qh_bundle owns immutable inference identity. These QH contracts are not widened into the package-root one-step API.

This package owns the stable imports for scorer protocols, actor-visible EVL adapters and feature banks, candidate encoders, predictions, and diagnostics.

Inputs must preserve actor/oracle separation. EVL fields, candidate poses, and semidense observations are actor-visible features. Oracle RRI labels, GT meshes, and target crops are supervision/evaluation assets.

1.1 Package guide

aria_nbv.vin owns actor-side candidate scoring. It contains the historical one-step VIN control and the modular target-conditioned finite-horizon scorer. Oracle RRI labels are supervision, not model inputs; Lightning owns losses, hard masking, and optimization.

1.2 Choose a Scorer

Scorer Use Output
VinModelV3 Target-agnostic one-step control used by the seminar pipeline. CORAL VinPrediction rows.
TargetSpatialQueryScorer Additive configurable target/ray EVL readout with optional gravity-aligned geometry and directional history. The same finite-horizon QhScoreOutput contract.
TargetVolumetricScorer Bounded evidence volume, multiscale 3D encoding, and candidate-local spatial attention. The same finite-horizon QhScoreOutput contract.
TargetFiniteHorizonScorer Actor-only conditional values for a target, state, candidate, budget, and requested horizon. QhScoreOutput with conditional_q, feasibility_logits, and optional CORAL auxiliaries.

Import the one-step stable surface from aria_nbv.vin. Import QH contracts from their leaf modules so the two training objectives remain explicit.

from aria_nbv.vin.models.target_finite_horizon import (
    TargetFiniteHorizonScorerConfig,
)

scorer = TargetFiniteHorizonScorerConfig(
    hidden_dim=128,
    max_horizon=4,
).setup_target()

output = scorer(actor, requested_horizon=None)
q = output.conditional_q
feasibility = output.feasibility_logits

requested_horizon=None queries actor.horizon_remaining. Explicit realized queries are scalar int64[B,S] values satisfying 1 <= h <= b_t; padding uses zero and unsupported values fail closed.

1.3 Finite-Horizon Architecture

The following diagram describes TargetFiniteHorizonScorer.

Code 
flowchart LR
  C["Candidate pose and candidate_mask"] --> P["Physical candidate trunk"]
  S["Current pose and S0/S1 scene"] --> P
  P --> F["Feasibility logits"]
  P --> Q["Candidate value query"]
  T["Target and candidate-relative relation"] --> Q
  X["Scene · target · history · budget · horizon tokens"] --> A["A0 MLP or A1 cross-attention"]
  Q --> A --> D["Regression or CORAL decoder"]
  D --> V["Conditional Q"]
  F --> O["QhScoreOutput"]
  V --> O
  M["Authoritative action_mask"] -. "used outside scorer" .-> L["Loss · backup · online selection"]

  classDef input fill:#D5E8D4,stroke:#82B366,color:#17202A,stroke-width:1.5px,rx:0,ry:0;
  classDef output fill:#F8CECC,stroke:#B85450,color:#17202A,stroke-width:1.5px,rx:0,ry:0;
  classDef compute fill:#E1D5E7,stroke:#9673A6,color:#17202A,stroke-width:1.5px,rx:8,ry:8;
  classDef data fill:#F5F5F5,stroke:#9E9E9E,color:#17202A,stroke-width:1.2px,rx:0,ry:0;
  class C,S,T,M input;
  class P,Q,A,D,F,V,L compute;
  class X data;
  class O output;

flowchart LR
  C["Candidate pose and candidate_mask"] --> P["Physical candidate trunk"]
  S["Current pose and S0/S1 scene"] --> P
  P --> F["Feasibility logits"]
  P --> Q["Candidate value query"]
  T["Target and candidate-relative relation"] --> Q
  X["Scene · target · history · budget · horizon tokens"] --> A["A0 MLP or A1 cross-attention"]
  Q --> A --> D["Regression or CORAL decoder"]
  D --> V["Conditional Q"]
  F --> O["QhScoreOutput"]
  V --> O
  M["Authoritative action_mask"] -. "used outside scorer" .-> L["Loss · backup · online selection"]

  classDef input fill:#D5E8D4,stroke:#82B366,color:#17202A,stroke-width:1.5px,rx:0,ry:0;
  classDef output fill:#F8CECC,stroke:#B85450,color:#17202A,stroke-width:1.5px,rx:0,ry:0;
  classDef compute fill:#E1D5E7,stroke:#9673A6,color:#17202A,stroke-width:1.5px,rx:8,ry:8;
  classDef data fill:#F5F5F5,stroke:#9E9E9E,color:#17202A,stroke-width:1.2px,rx:0,ry:0;
  class C,S,T,M input;
  class P,Q,A,D,F,V,L compute;
  class X data;
  class O output;

The implementation has two deliberately different branches:

  • The physical candidate trunk uses materialized candidate rows and candidate_mask. It is independent of the target, requested horizon, supervision, and authoritative action_mask, and feeds the feasibility head.
  • The conditional-Q branch adds the target token, candidate-relative target relation, scene summary, causal history, remaining budget, and requested horizon. A0 scores rows independently; A1 lets each candidate query state tokens without candidate-to-candidate attention.

Changing only action_mask must not change raw Q or feasibility. Lightning and online policy adapters apply that mask to losses, backups, and selection.

1.4 Representation Modules

The default configuration is intentionally compositional:

Axis Implementations
Scene S0 root moments; S1 selected-surface residual over S0.
History H0 masked mean; H1 ordered causal transformer.
State fusion A0 independent MLP; A1 candidate-to-state cross-attention.
Value decoding Direct regression; CORAL over a manifest-bound support.

The scorer preserves candidate permutation equivariance because candidates do not exchange information. Padding is zeroed. Materialized invalid rows receive finite conditional predictions, but those values are neither supervised nor deployable unless admitted by the external hard mask.

S1 is an implemented privileged experimental profile and remains unpromoted; deployable bundle publication currently admits actor-only CF0.

1.5 Spatial-query composition and inspection

TargetSpatialQueryScorerConfig lives in models.target_spatial_query. Its registered stages use existing TargetConfig.setup_target() factories. One pose encoder supplies all pose consumers. The physical branch reads root/current candidate poses and target-independent global evidence. Value queries additionally read a bounded target OBB lattice and candidate-to-target ray; ordered values and per-feature availability enter the model together. Unknown evidence remains distinct from measured zero and action validity. The existing fusion and decoder modules consume four shared state tokens: target, causal history, remaining budget and requested horizon.

Select QhSummaryReadoutConfig as readout for the global evidence control. target_geometry optionally selects a target-centred world-Z-up frame whose horizontal axis points toward the current camera. Active degenerate bearings fail explicitly. directional_history adds selected-prefix SH/radial coverage in that exact frame; the actor contract supplies no elapsed seconds, so model composition admits uniform weighting. The standalone history stage supports explicit caller-provided recency ages. Existing H0/H1 and S0/S1 controls retain their behavior and checkpoint identities.

QhExperimentConfig.scorer accepts the concrete model configurations. Old untagged configurations still select TargetFiniteHorizonScorer; the spatial model records model_kind="target_spatial_query_v1". Bundle loading validates that the selected model matches the manifest before loading strict weights. Lightning optimization has no feature-specific branch. This library support does not establish a training or policy advantage. Frustum/intersection readouts require additional calibrated evidence and remain outside this slice.

1.6 Volumetric evidence and learning

models.target_volumetric.TargetVolumetricScorerConfig specializes the spatial scorer by replacing its candidate readout. Pose encoding, physical feasibility, history, state fusion, value decoding, masks, and outputs keep their existing owners. The new model records model_kind="target_volumetric_v1".

The default map is a dense 48³ grid in the immutable root EVL frame, with a one-metre margin on each side of its extent. This bounds memory while allowing selected observations to contribute outside the original EVL chart. Evidence outside the expanded map is unsupported. The root features and their source availability are retained separately from selected-depth surface/free counts, observed support, and last-observed recency. Root counts describe valid image feature projections; selected counts describe admitted depth votes. Neither is calibrated uncertainty. Root free-space features retain their supplied semantics; they are not relabelled as measured ray traversal.

New commissioning uses target_volumetric_root_only_v2 or target_volumetric_selected_depth_v2 with readout.fusion.root_support_semantics="measured_evidence_v2": positive projection counts retain feature availability but cannot establish measured support. Occupied input and semidense measurements still supply root surface support. Historical v1 identities retain the explicit legacy support mode for checkpoint reproduction; a mismatched version and fusion policy is rejected. Free-input measurement promotion requires separately bound native-ray provenance.

root_only consumes CF0. selected_depth requires the admitted privileged CF-GT prefix and matching profile/representation identities. Each state’s volume is reconstructed from exactly its admitted observations j < s. Camera-Z depth and calibrated EFM3D camera/pose adapters determine surface and free evidence. An observation contributes at most once per voxel; repeating its identity is idempotent. Fusion returns a new evidence state and never updates root fields or persists learned activations between forward passes.

The encoder applies fine and half-resolution residual 3D convolutions, then fuses them at the original resolution. projection_only=True is the matched readout ablation. Each candidate attends independently to trilinearly sampled target-lattice and candidate-to-target-ray tokens with relative XYZ and region identity. Unknown in-map tokens remain readable; only map exclusion and padding remove tokens. These queries do not describe a calibrated full frustum.

Learned encoding, sampling with respect to feature values, attention, and the existing value path are differentiable. Evidence fusion, observation identity, projection membership, and hard masks are deterministic. The current scorer does not require gradients through sensing, RRI rewards, reconstruction, or camera selection. Native Torch and existing EFM3D utilities suffice; sparse convolution and VoxelBlockGrid are not dependencies of this implementation.

Selected-depth fitting uses the existing Lightning diagnostic lifecycle below; it does not authorize deployment of privileged inputs. Neither the model’s implementation nor one commissioning epoch establishes a policy advantage.

For rendering-neutral inspection, use vin.diagnostics.inspect_config before construction, inspect_model on an existing model, and capture_model around an explicitly requested ordinary forward pass:

from aria_nbv.vin.diagnostics import CaptureLimits, capture_model, inspect_config
from aria_nbv.vin.models.target_spatial_query import TargetSpatialQueryScorerConfig

config = TargetSpatialQueryScorerConfig(max_horizon=4)
structure = inspect_config(config)  # constructs no runtime model
scorer = config.setup_target()
with capture_model(scorer, limits=CaptureLimits(max_bytes=4096)) as capture:
    prediction = scorer(actor)
snapshots = capture.snapshots  # bounded detached values; hooks are removed

1.7 Regression and CORAL

Direct continuous regression is the canonical decoder and needs no support calibration. Select CORAL when the experiment has a frozen train-fitted support or a predeclared physical support whose calibration identity can be bound into the bundle. Legacy fixed support remains inspection-only and is not publishable:

from aria_nbv.vin.models.target_finite_horizon import (
    TargetFiniteHorizonScorerConfig,
)
from aria_nbv.vin.modules.qh_value_decoders import (
    QhCoralValueDecoderConfig,
    QhPredeclaredPhysicalCoralSupport,
)

support = QhPredeclaredPhysicalCoralSupport.create(
    source_population_digest="population-v1",
    ordered_input_digest="physical-rule-inputs-v1",
    physical_rule="symmetric-root-gain-support-v1",
    bin_edges=(-0.5, 0.5),
    bin_values=(-1.0, 0.0, 1.0),
)
config = TargetFiniteHorizonScorerConfig(
    max_horizon=4,
    value_decoder=QhCoralValueDecoderConfig(
        support=support,
        preinit_bias=False,
    ),
)

Quantile decoding is the next distributional family suggested by the current design, but is not implemented or part of the core architecture.

1.8 Detailed Contracts

  • Actor inputs: aria_nbv.data_handling.qh_data.views.QhActorTensors
  • Shared structured outputs: aria_nbv.vin.types.prediction
  • Concrete scorers: aria_nbv.vin.models.target_finite_horizon and target_spatial_query
  • Scene/history/fusion/decoder modules: aria_nbv.vin.modules.qh_*
  • Bundle identity: aria_nbv.vin.qh_bundle
  • Scientific estimand: finite-candidate value model
  • Human-facing API: generated reference

1.9 Verification

cd aria_nbv
uv run ruff check aria_nbv/vin
uv run pytest tests/vin/test_target_finite_horizon.py
uv run pytest tests/vin/test_qh_value_decoders.py tests/vin/test_qh_state_fusion.py

Include Lightning experiment tests whenever output, decoder, or bundle identity changes.

1.10 Attributes

Name Description
CandidateScorerConfig Config-as-factory type for candidate scorer architecture slots.

1.11 Classes

Name Description
CandidateScorer Protocol for trainable candidate scorers consumed by VIN Lightning.
CandidateScorerPrediction Minimal prediction attributes used by the current Lightning loss path.
EvlBackbone Run EVL and expose actor-visible feature volumes in their voxel frame.
EvlBackboneConfig Select the EVL model assets, device, and exported feature families.
EvlBackboneOutput Actor-visible EVL evidence exchanged with VIN scorers and caches.
FeaturePoolingResult Weighted point descriptors pooled over logged observations.
LearnableFourierFeatures Learnable Fourier Features (LFF) positional encoding.
LearnableFourierFeaturesConfig Config-as-factory wrapper for LearnableFourierFeatures.
PointFeatureBank Feature bank derived from logged actor-visible observations.
PointQueryPool Permutation-invariant descriptor statistics for a masked point query.
PoseEncoder Base interface for encoders of poses already in a reference frame.
PoseEncodingOutput Pose-encoding outputs for a pose expressed in a reference frame.
R6dLffPoseEncoder Encode reference-frame translation and rotation-6D through LFF.
R6dLffPoseEncoderConfig Configure reference-frame R6D plus LFF pose encoding.
TrajectoryEncoder Encode EFM trajectory poses with an R6D + LFF pose encoder.
TrajectoryEncoderConfig Configure per-frame pose encoding and sequence reduction.
TrajectoryEncodingOutput Trajectory encoding outputs.
VinModelV3 VIN-Core head for one-step RRI prediction.
VinModelV3Config Configuration for VinModelV3 (streamlined one-step VIN baseline).
VinPrediction VIN predictions for a candidate set.
VinV3ForwardDiagnostics Candidate-aligned intermediate tensors from VIN-Core forward passes.

1.11.1 CandidateScorer

vin.CandidateScorer()

Protocol for trainable candidate scorers consumed by VIN Lightning.

Implementations score each candidate pose row from actor-visible scene evidence and return a aria_nbv.vin.types.VinPrediction. The current Lightning objective also expects a CORAL head at head_coral so it can compute expected RRI proxies and initialize ordinal bin values.

1.11.1.1 Implementers

aria_nbv.vin.models.scene_myopic.VinModelV3 is the current one-step scorer. Future target-conditioned myopic scorers can satisfy this protocol when they still emit per-candidate ordinal logits. Finite-horizon QH scorers use their dedicated actor input, structured output, and Lightning objective instead of this protocol.

1.11.1.2 Attributes

Name Description
head_coral CORAL head used for ordinal probability and expected-value helpers.

1.11.1.3 Methods

Name Description
init_bin_values Initialize scorer-owned CORAL bin representatives.
forward Score candidate poses for one actor-visible snippet.
parameters Return trainable parameters, matching torch.nn.Module.parameters.
summarize_vin Return a debug summary for scorer inputs and modules.
1.11.1.3.1 init_bin_values
vin.CandidateScorer.init_bin_values(values, *, overwrite=False)

Initialize scorer-owned CORAL bin representatives.

1.11.1.3.2 forward
vin.CandidateScorer.forward(
    efm,
    *,
    candidate_poses_world_cam,
    reference_pose_world_rig,
    p3d_cameras,
    backbone_out=None,
)

Score candidate poses for one actor-visible snippet.

1.11.1.3.2.1 Parameters
Name Type Description Default
efm EfmSnippetView | VinSnippetView Actor-visible EFM/VIN snippet view containing semidense scene evidence. required
candidate_poses_world_cam PoseTW PoseTW["B N_q 12"] world←camera candidate poses. required
reference_pose_world_rig PoseTW PoseTW["B 12"] world←rig reference pose used to express candidates in the local frame. required
p3d_cameras PerspectiveCameras PyTorch3D camera batch aligned with candidate poses. required
backbone_out EvlBackboneOutput | None Optional cached EVL output; when omitted the scorer may compute it from efm if it owns a backbone. None
1.11.1.3.2.2 Returns
Name Type Description
VinPrediction Per-candidate logits, probabilities, expected scores, and optional
VinPrediction validity/coverage diagnostics.
1.11.1.3.3 parameters
vin.CandidateScorer.parameters(recurse=True)

Return trainable parameters, matching torch.nn.Module.parameters.

1.11.1.3.4 summarize_vin
vin.CandidateScorer.summarize_vin(
    batch,
    *,
    include_torchsummary=True,
    torchsummary_depth=3,
)

Return a debug summary for scorer inputs and modules.

1.11.2 CandidateScorerPrediction

vin.CandidateScorerPrediction()

Minimal prediction attributes used by the current Lightning loss path.

This protocol documents the structural subset of aria_nbv.vin.types.VinPrediction consumed by VinLightningModule. Optional coverage fields are read with getattr in the trainer, so they do not need to be present on future prediction dataclasses unless coverage weighting or diagnostics should use them.

1.11.2.1 Attributes

Name Description
logits Tensor["B N_q K-1", float32] CORAL threshold logits.
prob Tensor["B N_q K", float32] ordinal class probabilities.
expected_normalized Tensor["B N_q", float32] expected normalized ranking score.
candidate_valid Optional Tensor["B N_q", bool] scorer diagnostic validity mask.

1.11.3 EvlBackbone

vin.EvlBackbone(config)

Run EVL and expose actor-visible feature volumes in their voxel frame.

freeze controls gradients, but inference always executes under torch.no_grad and puts EVL in evaluation mode. The wrapper retains the resolved asset paths on config; returned tensors carry no independent checkpoint or config identity.

1.11.3.1 Methods

Name Description
forward Run EVL and return the feature volumes needed by VIN.
1.11.3.1.1 forward
vin.EvlBackbone.forward(efm)

Run EVL and return the feature volumes needed by VIN.

1.11.3.1.1.1 Parameters
Name Type Description Default
efm Mapping[str, Any] Raw EFM snippet dict (unbatched T×... or batched B×T×...). required
1.11.3.1.1.2 Returns
Name Type Description
EvlBackboneOutput Actor-visible EvlBackboneOutput whose voxel tensors are
EvlBackboneOutput Tensor["B C D H W"]-like fields aligned to
EvlBackboneOutput PoseTW["B 12"] world-from-voxel poses. Exact feature semantics
EvlBackboneOutput remain local to this config/checkpoint pair.

1.11.4 EvlBackboneConfig

vin.EvlBackboneConfig()

Select the EVL model assets, device, and exported feature families.

The resolved model_cfg and model_ckpt paths jointly determine the learned evidence semantics for one run. Paths are not immutable provenance: callers that persist outputs must also capture config and checkpoint hashes.

1.11.4.1 Attributes

Name Description
target_type Factory target for BaseConfig.setup_target.
paths Project path resolver.
model_cfg Resolved Hydra YAML used to instantiate efm3d.model.evl.EVL.
model_ckpt Resolved checkpoint containing an EVL state_dict payload.
device Torch device to run EVL on (auto-selects CUDA if available).
freeze Disable gradients for all EVL parameters when True.
features_mode Which EVL features to expose.
free_input_mode Source of free-space input: native EVL output or explicit derivation.

1.11.5 EvlBackboneOutput

vin.EvlBackboneOutput(
    t_world_voxel,
    voxel_extent,
    voxel_feat=None,
    occ_feat=None,
    obb_feat=None,
    occ_pr=None,
    occ_input=None,
    free_input=None,
    free_input_provenance=None,
    counts=None,
    counts_m=None,
    voxel_select_t=None,
    cent_pr=None,
    bbox_pr=None,
    clas_pr=None,
    cent_pr_nms=None,
    obbs_pr_nms=None,
    obb_pred=None,
    obb_pred_viz=None,
    obb_pred_sem_id_to_name=None,
    obb_pred_probs_full=None,
    obb_pred_probs_full_viz=None,
    pts_world=None,
    feat2d_upsampled=dict(),
    token2d=dict(),
)

Actor-visible EVL evidence exchanged with VIN scorers and caches.

The tensors are local to the EVL model configuration and checkpoint that produced them; they are learned evidence, not ground truth. This container does not embed source, config, or checkpoint hashes. Any durable offline payload must therefore pair it with provenance metadata that identifies the EVL source, resolved configuration, and checkpoint content.

1.11.5.1 Attributes

Name Type Description
t_world_voxel PoseTW PoseTW["B 12"] world←voxel pose for the voxel grid.
voxel_extent Tensor Tensor["6", float32] voxel grid extent in voxel frame [x_min,x_max,y_min,y_max,z_min,z_max] (meters).
voxel_feat Tensor | None Optional Tensor["B F D H W", float32] raw voxel features from the 3D backbone.
occ_feat Tensor | None Optional Tensor["B C D H W", float32] neck features for occupancy.
obb_feat Tensor | None Optional Tensor["B C D H W", float32] neck features for OBB detection.
occ_pr Tensor | None Optional Tensor["B 1 D H W", float32] EVL occupancy probability.
occ_input Tensor | None Optional Tensor["B 1 D H W", float32] voxelized occupied evidence from input points.
free_input Tensor | None Optional Tensor["B 1 D H W", float32] voxelized free-space evidence.
free_input_provenance FreeInputProvenance | None Native EVL output or the explicit derived construction.
counts Tensor | None Optional Tensor["B D H W", int64] per-voxel observation counts.
counts_m Tensor | None Optional Tensor["B D H W", int64] masked/debug variant of counts.
voxel_select_t Tensor | None Optional Tensor["B 1", int64] frame index anchoring the voxel grid.
cent_pr Tensor | None Optional Tensor["B 1 D H W", float32] centerness probabilities.
bbox_pr Tensor | None Optional Tensor["B 7 D H W", float32] bounding box regressions.
clas_pr Tensor | None Optional Tensor["B K D H W", float32] class probabilities.
cent_pr_nms Tensor | None Optional Tensor["B 1 D H W", float32] centerness after NMS.
obbs_pr_nms ObbTW | None Optional ObbTW["B M 34"] OBB predictions after NMS (voxel frame).
obb_pred ObbTW | None Optional ObbTW["B M 34"] OBB predictions in snippet coordinates.
obb_pred_viz ObbTW | None Optional ObbTW["B M 34"] visualization OBB predictions in snippet coordinates.
obb_pred_sem_id_to_name dict[int, str] | None Optional sparse semantic id to class-name map aligned with EVL taxonomy.
obb_pred_probs_full list[Tensor] | None Optional list of per-OBB class probability tensors.
obb_pred_probs_full_viz list[Tensor] | None Optional list of per-OBB class probability tensors for visualization.
pts_world Tensor | None Optional Tensor["B (D·H·W) 3", float32] world-space voxel centers.
feat2d_upsampled dict[str, Tensor] Per-stream 2D feature maps keyed by stream name.
token2d dict[str, Tensor] Per-stream 2D tokens keyed by stream name.

1.11.5.2 Methods

Name Description
to_serializable Serialize this backbone output into a cache-friendly CPU payload.
from_serializable Reconstruct one backbone output from a serialized payload.
to Move all tensors to the specified device.
1.11.5.2.1 to_serializable
vin.EvlBackboneOutput.to_serializable(include_fields=None)

Serialize this backbone output into a cache-friendly CPU payload.

1.11.5.2.1.1 Parameters
Name Type Description Default
include_fields set[str] | None Optional dataclass field names to include. When omitted, all fields are serialized. None
1.11.5.2.1.2 Returns
Name Type Description
dict[str, Any] CPU-only payload suitable for immutable offline-store records.
1.11.5.2.2 from_serializable
vin.EvlBackboneOutput.from_serializable(payload, *, device, include_fields=None)

Reconstruct one backbone output from a serialized payload.

1.11.5.2.2.1 Parameters
Name Type Description Default
payload dict[str, Any] Serialized payload produced by to_serializable. required
device torch.device Destination device for tensors and wrappers. required
include_fields set[str] | None Optional subset of fields to decode. None
1.11.5.2.2.2 Returns
Name Type Description
'EvlBackboneOutput' Reconstructed backbone output.
1.11.5.2.3 to
vin.EvlBackboneOutput.to(device)

Move all tensors to the specified device.

1.11.6 FeaturePoolingResult

vin.FeaturePoolingResult(features, valid_mask, valid_frame_count, weight_sum)

Weighted point descriptors pooled over logged observations.

1.11.6.1 Attributes

Name Description
features Tensor["B N_p C", float32] weighted descriptor mean per point.
valid_mask Tensor["B N_p", bool] points supported by at least one frame.
valid_frame_count Tensor["B N_p", int64] valid logged frames per point.
weight_sum Tensor["B N_p", float32] pooling-weight sum before epsilon.

1.11.7 LearnableFourierFeatures

vin.LearnableFourierFeatures(config)

Learnable Fourier Features (LFF) positional encoding.

This module maps continuous inputs \(x \in \mathbb{R}^D\) into a learned feature space by:

  1. learning a projection matrix \(W_r\),
  2. applying sinusoidal features, and
  3. mapping them through a small MLP.

Compared to fixed random Fourier features, the learned projection and the MLP allow the encoding to adapt to the downstream task. This is a learned coordinate-feature inductive bias; the arbitrary projection and MLP do not enforce translation, rotation, or SE(3) equivariance.

1.11.7.1 Attributes

Name Description
out_dim Return the emitted feature dimension, including raw inputs when enabled.

1.11.7.2 Methods

Name Description
forward Encode vectors with learned sinusoidal features.
1.11.7.2.1 forward
vin.LearnableFourierFeatures.forward(x)

Encode vectors with learned sinusoidal features.

1.11.7.2.1.1 Parameters
Name Type Description Default
x Tensor Tensor["... D"] input vectors with D == input_dim. required
1.11.7.2.1.2 Returns
Name Type Description
Tensor Tensor["... E"] encoded vectors, optionally concatenated with
Tensor the raw input when LearnableFourierFeaturesConfig.include_input
Tensor is enabled.

1.11.8 LearnableFourierFeaturesConfig

vin.LearnableFourierFeaturesConfig()

Config-as-factory wrapper for LearnableFourierFeatures.

1.11.8.1 Attributes

Name Description
target_type Factory target for aria_nbv.configs.base.BaseConfig.setup_target.
out_dim Return the width emitted by this configuration.
input_dim Input dimensionality (default: 6 for SE(3) relative pose as translation + so(3) log).
fourier_dim Fourier feature dimensionality (must be even).
hidden_dim Hidden dimension of the internal MLP.
output_dim Output positional encoding dimensionality (per input vector).
gamma Initialization scale (paper parameter \(\gamma\)).
include_input Concatenate raw inputs to the learned encoding when True.

1.11.9 PointFeatureBank

vin.PointFeatureBank(
    points_world,
    features,
    valid_mask,
    valid_frame_count,
    weight_sum,
    per_frame_valid,
    source_frame_indices,
    feature_source,
    source_role='actor_visible',
    compression_id='raw',
    point_support=None,
)

Feature bank derived from logged actor-visible observations.

The dataclass validates provenance at construction but is not frozen; callers that require immutable cache records must enforce immutability at the storage boundary.

Descriptor values are learned-source-local rather than globally comparable. feature_source and compression_id are human-readable provenance labels, not content hashes. Persistent EVL-derived banks must be paired by their cache owner with source-config and checkpoint hashes.

1.11.9.1 Attributes

Name Description
points_world Tensor["B N_p 3", float32] world-frame point coordinates in metres.
features Tensor["B N_p C", float32] logged-view descriptors pooled per point.
valid_mask Tensor["B N_p", bool] mask for points with descriptor support.
valid_frame_count Tensor["B N_p", int64] valid logged-frame samples per point.
weight_sum Tensor["B N_p", float32] pooling-weight sum before epsilon.
per_frame_valid Tensor["B T N_p", bool] per-frame projection-valid mask.
source_frame_indices Tensor["B T", int64] logged source-frame indices.
feature_source Approved actor-visible source label; not a checkpoint content hash.
source_role Provenance role; actor banks accept only "actor_visible".
compression_id Descriptor compression label; not a projection-matrix content hash.
point_support Optional point/sample weights broadcast by the multiview pooler.

1.11.9.2 Methods

Name Description
validate_actor_visible Reject oracle, GT, future-candidate, or unapproved feature sources.
1.11.9.2.1 validate_actor_visible
vin.PointFeatureBank.validate_actor_visible()

Reject oracle, GT, future-candidate, or unapproved feature sources.

1.11.10 PointQueryPool

vin.PointQueryPool(mean, maximum, std, count, valid_mask)

Permutation-invariant descriptor statistics for a masked point query.

1.11.10.1 Attributes

Name Description
mean Tensor["B C", float32] selected-point descriptor mean.
maximum Tensor["B C", float32] selected-point elementwise maximum.
std Tensor["B C", float32] selected-point population deviation.
count Tensor["B", int64] selected-point count.
valid_mask Tensor["B", bool] non-empty query-support mask.

1.11.11 PoseEncoder

vin.PoseEncoder()

Base interface for encoders of poses already in a reference frame.

1.11.11.1 Attributes

Name Description
pose_encoder_lff Optional LFF submodule for diagnostics (None for SH-only encoders).
out_dim Return the final pose-embedding feature width.

1.11.11.2 Methods

Name Description
encode Encode a pose expressed in the reference frame.
forward Encode a reference-frame pose through PyTorch’s module call interface.
1.11.11.2.1 encode
vin.PoseEncoder.encode(pose_rig)

Encode a pose expressed in the reference frame.

1.11.11.2.2 forward
vin.PoseEncoder.forward(pose_rig)

Encode a reference-frame pose through PyTorch’s module call interface.

1.11.11.2.2.1 Parameters
Name Type Description Default
pose_rig PoseTW PoseTW with floating storage shape ... 12; the caller selects the reference frame and translations are metres. required
1.11.11.2.2.2 Returns
Name Type Description
PoseEncodingOutput The same differentiable PoseEncodingOutput as :meth:encode,
PoseEncodingOutput preserving leading pose axes. Calling the module also runs registered
PoseEncodingOutput hooks; no extra parameters or transformation are introduced here.

1.11.12 PoseEncodingOutput

vin.PoseEncodingOutput(
    center_m,
    pose_vec,
    pose_enc,
    center_dir=None,
    forward_dir=None,
    radius_m=None,
    view_alignment=None,
)

Pose-encoding outputs for a pose expressed in a reference frame.

1.11.12.1 Attributes

Name Type Description
center_m Tensor Tensor["... 3", float32] translation in the reference frame.
pose_vec Tensor Tensor["... D", float32] pose vector fed into the encoder.
pose_enc Tensor Tensor["... E", float32] encoded pose embedding.
center_dir Tensor | None Optional Tensor["... 3", float32] unit direction to center.
forward_dir Tensor | None Optional Tensor["... 3", float32] forward direction in ref frame.
radius_m Tensor | None Optional Tensor["... 1", float32] radius \|\|t\|\|.
view_alignment Tensor | None Optional Tensor["... 1", float32] dot <f, -u>.

1.11.13 R6dLffPoseEncoder

vin.R6dLffPoseEncoder(config)

Encode reference-frame translation and rotation-6D through LFF.

The representation is continuous for common rotations but is not an SE(3)-equivariant map: translation coordinates and rotation-matrix columns are interpreted in the caller-supplied reference rig frame.

1.11.13.1 Attributes

Name Description
pose_encoder_lff Registered shared Fourier encoder; follows module mode, device and checkpoint lifecycle.
out_dim Return the configured LFF pose-embedding width.

1.11.13.2 Methods

Name Description
encode Encode poses in the reference rig frame.
1.11.13.2.1 encode
vin.R6dLffPoseEncoder.encode(pose_rig)

Encode poses in the reference rig frame.

1.11.13.2.1.1 Parameters
Name Type Description Default
pose_rig PoseTW PoseTW["... 12"] pose in reference rig frame. required
1.11.13.2.1.2 Returns
Name Type Description
PoseEncodingOutput PoseEncodingOutput containing translation, pose vector, and embedding.

1.11.14 R6dLffPoseEncoderConfig

vin.R6dLffPoseEncoderConfig()

Configure reference-frame R6D plus LFF pose encoding.

1.11.14.1 Attributes

Name Description
target_type Factory target for aria_nbv.configs.base.BaseConfig.setup_target.
out_dim Return the complete pose-embedding width seen by consumers.
kind Discriminator for pose-encoder selection.
pose_encoder_lff LFF encoder for [t, r6d] pose vectors (input_dim=9).
pose_scale_init Initial per-group scale for translation and rotation (t, r6d).
pose_scale_learnable Whether pose scaling parameters are learned.
pose_scale_eps Numerical epsilon for pose scaling.

1.11.15 TrajectoryEncoder

vin.TrajectoryEncoder(config)

Encode EFM trajectory poses with an R6D + LFF pose encoder.

1.11.15.1 Attributes

Name Description
out_dim Return the per-frame and pooled trajectory embedding width.

1.11.15.2 Methods

Name Description
encode_poses Encode a batch of trajectory poses.
forward Encode a trajectory’s world←rig poses.
1.11.15.2.1 encode_poses
vin.TrajectoryEncoder.encode_poses(poses)

Encode a batch of trajectory poses.

1.11.15.2.1.1 Parameters
Name Type Description Default
poses PoseTW PoseTW["B F 12"] or PoseTW["F 12"] trajectory poses. required
1.11.15.2.1.2 Returns
Name Type Description
TrajectoryEncodingOutput TrajectoryEncodingOutput with per-frame encodings and pooled embedding.
1.11.15.2.2 forward
vin.TrajectoryEncoder.forward(trajectory)

Encode a trajectory’s world←rig poses.

1.11.15.2.2.1 Parameters
Name Type Description Default
trajectory EfmTrajectoryView aria_nbv.data_handling.ase_efm.views.EfmTrajectoryView. required
1.11.15.2.2.2 Returns
Name Type Description
TrajectoryEncodingOutput TrajectoryEncodingOutput with per-frame encodings and pooled embedding.

1.11.16 TrajectoryEncoderConfig

vin.TrajectoryEncoderConfig()

Configure per-frame pose encoding and sequence reduction.

1.11.16.1 Attributes

Name Description
target_type Factory target for aria_nbv.configs.base.BaseConfig.setup_target.
pose_encoder Pose encoder configuration (R6D + LFF).
pool_mode Pooling mode over frames: mean, final, or none.

1.11.17 TrajectoryEncodingOutput

vin.TrajectoryEncodingOutput(per_frame, pooled)

Trajectory encoding outputs.

1.11.17.1 Attributes

Name Type Description
per_frame PoseEncodingOutput PoseEncodingOutput for each frame.
pooled Tensor | None Tensor["B E", float32] pooled trajectory embedding (or None).

1.11.18 VinModelV3

vin.VinModelV3(config)

VIN-Core head for one-step RRI prediction.

VIN v3 focuses on pose encoding, compact voxel evidence, and semidense projection stats, while enforcing fail-fast contracts to avoid silent collapse. It ranks candidates for immediate RRI; bounded rollout values such as \(Q_H\) are separate thesis models trained on rollout replay.

Candidate-set symmetry is structural: learned blocks use row-wise maps, candidate queries over shared scene/trajectory tokens, and symmetric GroupNorm statistics across rows. There are no candidate-index embeddings or order-sensitive candidate operations. In deterministic evaluation, jointly permuting candidate poses and aligned camera rows therefore permutes every N_q output the same way; stochastic training dropout preserves only distributional equivariance. This is not graph-isomorphism invariance because the model constructs no candidate graph. Its reference-frame pose and positional encodings are coordinate-conditioned inductive biases and do not enforce SE(3) equivariance.

1.11.18.1 Attributes

Name Description
pose_encoder_lff Return the LFF sub-encoder when present (else None).

1.11.18.2 Methods

Name Description
train Set training mode and discard inference-only prepared scene state.
forward Score an aligned candidate set with cached actor-visible evidence.
forward_with_debug Run VIN v3 forward pass and return intermediate tensors.
init_bin_values Initialize learnable bin representatives for CORAL expectation.
summarize_vin Summarize VIN v3 inputs/outputs for a single oracle-labeled batch.
1.11.18.2.1 train
vin.VinModelV3.train(mode=True)

Set training mode and discard inference-only prepared scene state.

1.11.18.2.2 forward
vin.VinModelV3.forward(
    efm,
    candidate_poses_world_cam,
    reference_pose_world_rig,
    p3d_cameras,
    backbone_out=None,
)

Score an aligned candidate set with cached actor-visible evidence.

1.11.18.2.2.1 Parameters
Name Type Description Default
efm EfmSnippetView | VinSnippetView EFM snippet view or VIN snippet view for the current snippet. required
candidate_poses_world_cam PoseTW PoseTW["B N_q 12"] world-from-camera poses. required
reference_pose_world_rig PoseTW PoseTW["B 12"] world-from-rig pose. required
p3d_cameras PerspectiveCameras PyTorch3D cameras aligned with the flattened B * N_q candidate rows. required
backbone_out EvlBackboneOutput | None Required cached actor-visible EVL evidence. None is accepted by the signature only to produce a fail-fast error. None
1.11.18.2.2.2 Returns
Name Type Description
VinPrediction VinPrediction with Tensor["B N_q K-1", float32] CORAL logits,
VinPrediction Tensor["B N_q K", float32] probabilities, and aligned expected
VinPrediction scores and diagnostic validity fields.
1.11.18.2.3 forward_with_debug
vin.VinModelV3.forward_with_debug(
    efm,
    candidate_poses_world_cam,
    reference_pose_world_rig,
    p3d_cameras,
    backbone_out=None,
)

Run VIN v3 forward pass and return intermediate tensors.

1.11.18.2.3.1 Parameters
Name Type Description Default
efm EfmSnippetView | VinSnippetView EFM snippet view or VIN snippet view for the current snippet. required
candidate_poses_world_cam PoseTW PoseTW["B N_q 12"] world-from-camera poses. required
reference_pose_world_rig PoseTW PoseTW["B 12"] world-from-rig pose. required
p3d_cameras PerspectiveCameras PyTorch3D cameras aligned with B * N_q rows. required
backbone_out EvlBackboneOutput | None Required cached actor-visible EVL evidence. None
1.11.18.2.3.2 Returns
Name Type Description
tuple[VinPrediction, VinV3ForwardDiagnostics] Prediction and B/N_q-aligned intermediate tensors.
1.11.18.2.4 init_bin_values
vin.VinModelV3.init_bin_values(values, *, overwrite=False)

Initialize learnable bin representatives for CORAL expectation.

1.11.18.2.4.1 Parameters
Name Type Description Default
values Tensor Tensor["K"] target bin representatives (e.g., bin means). required
overwrite bool If True, overwrite existing bin values. False
1.11.18.2.5 summarize_vin
vin.VinModelV3.summarize_vin(
    batch,
    *,
    include_torchsummary=True,
    torchsummary_depth=3,
)

Summarize VIN v3 inputs/outputs for a single oracle-labeled batch.

This is intended for quick sanity checks when validating the streamlined baseline against sweep-derived expectations.

1.11.18.2.5.1 Parameters
Name Type Description Default
batch VinOracleBatch Oracle-labeled batch to inspect. required
include_torchsummary bool Whether to include a torchsummary block. True
torchsummary_depth int Depth for torchsummary. 3
1.11.18.2.5.2 Returns
Name Type Description
str Human-readable summary string.

1.11.19 VinModelV3Config

vin.VinModelV3Config()

Configuration for VinModelV3 (streamlined one-step VIN baseline).

1.11.19.1 Attributes

Name Description
target_type Factory target for BaseConfig.setup_target (config-as-factory).
backbone Optional EVL config kept for caller-side materialization of voxel features.
pose_encoder Pose encoder configuration (R6D + LFF; stable relative pose encoding).
pos_grid_encoder_lff LFF encoder for XYZ voxel position keys used by global pooling.
head_hidden_dim Hidden dimension for the scorer MLP (optuna favored compact heads).
head_num_layers Number of MLP layers before the CORAL layer (best trials used 1).
head_dropout Dropout probability in the MLP (sweep best used near-zero dropout).
num_classes Number of ordinal bins (VIN-NBV uses 15 for sweep comparability).
coral_preinit_bias Pre-initialize CORAL biases for faster, more stable ordinal learning.
field_dim Channel dimension of the projected voxel field (compact by design).
field_gn_groups Requested GroupNorm groups (clamped to a divisor of field_dim) for stability.
semidense_proj_grid_size Grid size for semidense projection stats (higher for tiny-CNN cues).
semidense_proj_max_points Maximum semidense points used for projection stats (sweep best used 4096).
semidense_cnn_enabled Whether to encode a tiny 2D CNN over the semidense projection grid.
semidense_cnn_channels Hidden channel width for the semidense projection CNN.
semidense_cnn_out_dim Output feature dimension of the semidense projection CNN.
use_traj_encoder Whether to encode snippet trajectories and append trajectory context.
traj_encoder Optional trajectory encoder for snippet rig poses (R6D + LFF).
use_voxel_valid_frac_gate Deprecated: voxel gate removed. Keep False (use FiLM only).
semidense_obs_count_min Global minimum of semidense observation count n_obs (cache summary).
semidense_obs_count_max Global maximum of semidense observation count n_obs (cache summary).
semidense_obs_count_p95 Global 95th percentile of semidense observation count n_obs (cache summary).
semidense_obs_count_mean Global mean of semidense observation count n_obs (cache summary).
semidense_obs_count_std Global standard deviation of semidense observation count n_obs (cache summary).
semidense_inv_dist_std_min Global minimum of semidense inverse depth std 1/sigma_d (cache summary).
semidense_inv_dist_std_max Global maximum of semidense inverse depth std 1/sigma_d (cache summary).
semidense_inv_dist_std_p95 Global 95th percentile of semidense inverse depth std 1/sigma_d (cache summary).
semidense_inv_dist_std_mean Global mean of semidense inverse depth std 1/sigma_d (cache summary).
semidense_inv_dist_std_std Global standard deviation of semidense inverse depth std 1/sigma_d (cache summary).
apply_cw90_correction Undo rotate_yaw_cw90 on poses (requires CW90-corrected cameras).
global_pool_grid_size Target grid size for pose-conditioned global pooling (best trials used ~5).
scene_field_channels Ordered list of scene-field channels to include in the voxel field.

1.11.20 VinPrediction

vin.VinPrediction(
    logits,
    prob,
    expected,
    expected_normalized,
    candidate_valid,
    voxel_valid_frac=None,
    semidense_candidate_vis_frac=None,
)

VIN predictions for a candidate set.

This is the primary output of aria_nbv.vin.models.scene_myopic.VinModelV3. It is consumed by the Lightning training loop (loss + metrics) and by downstream NBV selection (ranking candidates by predicted improvement). The expected score is a learned one-step ranking proxy; rollout-level endpoint metrics and cumulative target RRI are produced by rollout stores and oracle re-scoring, not by this container.

The candidate axis is row-aligned with the input poses and PyTorch3D camera batch. VinModelV3 uses row-wise maps, shared-token queries, and symmetric normalization without candidate-index embeddings. A joint permutation of aligned inputs therefore produces the same permutation of every N_q output in deterministic evaluation; training-time dropout preserves this symmetry only in distribution.

Typical usage in training (see aria_nbv/lightning/one_step/module.py): - logits / prob: CORAL ordinal loss and optional auxiliary losses. - expected_normalized: correlation/top-k metrics and candidate ranking proxy. - voxel_valid_frac / semidense_candidate_vis_frac: optional scheduled coverage reweighting of the loss + diagnostics. - candidate_valid: conservative validity heuristic used for logging and optional filtering in analysis/visualization.

1.11.20.1 Attributes

Name Description
logits Tensor["B N_q K-1", float32] CORAL threshold logits.
prob Tensor["B N_q K", float32] class probabilities from the logits.
expected Tensor["B N_q", float32] expected class value in [0, K-1].
expected_normalized Tensor["B N_q", float32] expected value normalized to [0, 1].
candidate_valid Tensor["B N_q", bool] candidate validity diagnostic.
voxel_valid_frac Tensor["B N_q", float32] candidate voxel-coverage proxy, if available.
semidense_candidate_vis_frac Tensor["B N_q", float32] candidate semidense-visibility proxy, if available.
semidense_valid_frac Read-only compatibility alias for semidense_candidate_vis_frac.

1.11.21 VinV3ForwardDiagnostics

vin.VinV3ForwardDiagnostics(
    backbone_out,
    candidate_center_rig_m,
    pose_enc,
    pose_vec,
    field_in,
    field,
    global_feat,
    candidate_valid,
    feats,
    voxel_valid_frac=None,
    semidense_candidate_vis_frac=None,
    pos_grid=None,
    semidense_proj=None,
    semidense_grid_feat=None,
    voxel_proj=None,
    traj_feat=None,
    traj_ctx=None,
    traj_pose_vec=None,
    traj_pose_enc=None,
)

Candidate-aligned intermediate tensors from VIN-Core forward passes.

1.11.21.1 Attributes

Name Description
backbone_out EVL backbone outputs used to build the scene field.
candidate_center_rig_m Tensor["B N_q 3", float32] candidate centers in reference-rig metres.
pose_enc Tensor["B N_q E_pose", float32] pose encoder output.
pose_vec Tensor["B N_q D_pose", float32] pose vector fed into the encoder.
field_in Tensor["B C_in D H W", float32] Raw scene field before projection.
field Tensor["B C_out D H W", float32] Projected scene field.
global_feat Tensor["B N_q C_global", float32] pose-conditioned global features.
candidate_valid Tensor["B N_q", bool] diagnostic candidate-valid mask.
feats Tensor["B N_q F", float32] concatenated VIN candidate features.
voxel_valid_frac Tensor["B N_q", float32] candidate voxel-coverage proxy, if computed.
semidense_candidate_vis_frac Tensor["B N_q", float32] candidate semidense visibility, if computed.
semidense_valid_frac Read-only compatibility alias for semidense_candidate_vis_frac.
pos_grid Tensor["B 3 D H W", float32] Normalized voxel position grid (if computed).
semidense_proj Tensor["B N_q C_proj", float32] semidense projection features.
semidense_grid_feat Tensor["B N_q C_cnn", float32] CNN projection-grid features.
voxel_proj Tensor["B N_q C_proj", float32] voxel-projection features.
traj_feat Tensor["B F_traj", float32] Pooled trajectory embedding (if enabled).
traj_ctx Tensor["B N_q F_pose", float32] trajectory context per candidate query.
traj_pose_vec Tensor["B T D_v", float32] Per-frame trajectory pose vectors.
traj_pose_enc Tensor["B T F_traj", float32] Per-frame trajectory pose encodings.

1.12 Functions

Name Description
compress_point_features Apply an explicit descriptor compression transform.
pool_multiview_point_features Pool per-frame point descriptors with projection-valid weights.
pool_point_query Compute permutation-invariant masked point-pool summaries.
sample_logged_image_features_at_world_points Sample logged image features at semidense or fused world points.
validate_actor_feature_provenance Validate that a feature source can be consumed by actor-side models.

1.12.1 compress_point_features

vin.compress_point_features(features, *, projection=None, output_dim=None)

Apply an explicit descriptor compression transform.

1.12.1.1 Parameters

Name Type Description Default
features Tensor Tensor[..., C] point descriptors. required
projection Tensor | None Optional Tensor["C D"] projection matrix. None
output_dim int | None Optional leading channel count when using a simple slice. None

1.12.1.2 Returns

Name Type Description
Tensor Compressed descriptors.

1.12.2 pool_multiview_point_features

vin.pool_multiview_point_features(
    sampled_features,
    valid_mask,
    *,
    point_weights=None,
    eps=1e-06,
)

Pool per-frame point descriptors with projection-valid weights.

1.12.2.1 Parameters

Name Type Description Default
sampled_features Tensor Tensor["B T N C"] sampled logged-frame features. required
valid_mask Tensor Tensor["B T N", bool] projection-valid samples. required
point_weights Tensor | None Optional weights shaped N, T, B N, B T, T N, or B T N. Typical values encode point uncertainty, support, recency, or logged-view quality. None
eps float Positive denominator guard. 1e-06

1.12.2.2 Returns

Name Type Description
FeaturePoolingResult Weighted means and validity diagnostics.

1.12.3 pool_point_query

vin.pool_point_query(point_features, point_mask, *, eps=1e-06)

Compute permutation-invariant masked point-pool summaries.

Mean, maximum, population standard deviation, and count are symmetric in the N_p axis. Jointly permuting point_features and point_mask therefore leaves every output unchanged. This is set-permutation invariance, not a graph-isomorphism contract; no edges or message passing are constructed here.

1.12.3.1 Parameters

Name Type Description Default
point_features Tensor Tensor["B N_p C", float32] point descriptors. required
point_mask Tensor Tensor["B N_p", bool] points included in the query. required
eps float Positive denominator guard for mean and std. 1e-06

1.12.3.2 Returns

Name Type Description
PointQueryPool Mean, max, std, count, and empty-support mask.

1.12.4 sample_logged_image_features_at_world_points

vin.sample_logged_image_features_at_world_points(
    points_world,
    feat2d,
    cameras,
    t_world_camera,
    point_weights=None,
    source_frame_indices=None,
    feature_source='efm3d_feat2d_upsampled',
    source_role='actor_visible',
    compression_projection=None,
    compression_output_dim=None,
    compression_id='raw',
    eps=1e-06,
    warn=False,
)

Sample logged image features at semidense or fused world points.

1.12.4.1 Parameters

Name Type Description Default
points_world Tensor Tensor["N_p 3", float32] or Tensor["B N_p 3", float32] world-frame points in metres. required
feat2d Tensor Logged Tensor["B T C H W", float32] or Tensor["T C H W", float32] feature maps. required
cameras CameraTW CameraTW with shape B T or T. required
t_world_camera PoseTW PoseTW with shape B T or T. required
point_weights Tensor | None Optional point/sample weights for pooling. None
source_frame_indices Tensor | None Optional frame ids shaped B T or T. None
feature_source str Approved actor-visible source label. This label does not replace model-config or checkpoint hashes for persisted banks. 'efm3d_feat2d_upsampled'
source_role str Actor/oracle provenance role. 'actor_visible'
compression_projection Tensor | None Optional projection matrix for descriptors. None
compression_output_dim int | None Optional slice dimension for descriptors. None
compression_id str Stable compression label; content-address the projection separately when durable identity matters. 'raw'
eps float Positive denominator guard. 1e-06
warn bool Forward low-validity warnings from EFM3D sample_images. False

1.12.4.2 Returns

Name Type Description
PointFeatureBank Actor-visible point descriptors and provenance masks.

1.12.5 validate_actor_feature_provenance

vin.validate_actor_feature_provenance(
    feature_source,
    source_role='actor_visible',
)

Validate that a feature source can be consumed by actor-side models.

1.12.5.1 Parameters

Name Type Description Default
feature_source str Stable source id such as "efm3d_feat2d_upsampled". required
source_role str Provenance role. Actor inputs currently accept only "actor_visible". 'actor_visible'

1.12.5.2 Raises

Name Type Description
ValueError If the source is oracle/GT/counterfactual-only evidence.

1.12.5.3 Notes

Validation is label-based. Passing this check does not authenticate a config, checkpoint, compression projection, or cached tensor payload.

Back to top