1 data_handling.ase_efm

data_handling.ase_efm

Raw ASE/EFM data interfaces used by aria_nbv.data_handling.

This module provides the stable raw snippet layer for the data-handling package. It keeps the existing raw dataset and typed view implementations intact while presenting a single import surface for:

  • the iterable raw ASE/EFM dataset,
  • typed raw and VIN snippet views, and
  • the worker-local snippet loader used for live snippet attachment.

1.1 Package guide

aria_nbv.data_handling.ase_efm streams tensorized ASE snippets from ATEK WebDataset shards and presents them as typed EFM views. It is the live input used by candidate generation, immutable VIN-store construction, and small rollout-generation smoke tests.

ATEK shards carry calibrated observations, trajectory, and MPS semidense points. Optional ASE meshes and GT annotations are attached as oracle/evaluation assets and must not become ordinary actor inputs.

1.2 Read Snippets

from aria_nbv.data_handling.ase_efm import AseEfmDatasetConfig

dataset = AseEfmDatasetConfig(
    scene_ids=["81286"],
    load_meshes=True,
    require_mesh=True,
    device="cpu",
).setup_target()

snippet = next(iter(dataset))
print(snippet.scene_id, snippet.snippet_id)

AseEfmDatasetConfig resolves scene, shard, and sample filters; applies the ATEK-to-EFM taxonomy; and optionally pairs ASE meshes. EfmSnippetView exposes typed camera, trajectory, semidense-point, OBB, and GT surfaces without adding a parallel field schema. EfmSnippetLoader is the worker-local adapter for reattaching a known raw snippet to an immutable store row.

Use the downloader before constructing the dataset:

cd aria_nbv
uv run nbv-downloader -m list -c efm -n 10
uv run nbv-downloader -m download -c efm --ns 1 --max-shards 1

Detailed field, frame, and padding contracts live in the module docstrings and generated API reference.

1.3 Verification

cd aria_nbv
uv run ruff check aria_nbv/data_handling/ase_efm
uv run pytest tests/data_handling/test_efm_dataset_snippet.py \
  tests/data_handling/test_efm_snippet_loader.py

1.4 Classes

Name Description
AseEfmDataset Stream zero-copy EFM views from ATEK shards with optional ASE meshes.
AseEfmDatasetConfig Configure the canonical ATEK-shard to EFM-snippet streaming path.
EfmCameraView Expose one calibrated Aria camera stream view without copying EFM tensors.
EfmGTView Parse nested gt_data/efm_gt annotations for evaluation-only access.
EfmObbView Expose snippet-level GT boxes in the EFM ObbTW layout.
EfmPointsView Expose actor-visible MPS semidense reconstruction evidence.
EfmSnippetLoader Resolve source snippets through worker-owned, per-scene dataset caches.
EfmSnippetView Own the typed boundary around one adapted ATEK/ASE snippet.
EfmTrajectoryView Expose the MPS/EFM world-from-rig trajectory for one snippet.

1.4.1 AseEfmDataset

data_handling.ase_efm.AseEfmDataset(config)

Stream zero-copy EFM views from ATEK shards with optional ASE meshes.

Each iteration adapts one WebDataset sample into the consumed EFM key families, infers stable scene/snippet lineage, and attaches processed mesh tensors when configured. The iterable owns per-instance mesh memoization; yielded EfmSnippetView objects retain the backing EFM dictionary. Meshes never become actor-visible observations.

1.4.2 AseEfmDatasetConfig

data_handling.ase_efm.AseEfmDatasetConfig()

Configure the canonical ATEK-shard to EFM-snippet streaming path.

The config resolves scene/shard/sample filters, the ATEK-to-EFM taxonomy, and optional ASE mesh pairing before constructing AseEfmDataset. EfmModelAdaptor owns flattened-key remapping; this config does not introduce a parallel schema.

1.4.2.1 Attributes

Name Description
cache_exclude_fields Fields omitted from cache snapshots because they are large or derived.
target_type Factory target for BaseConfig.setup_target.
paths Project path resolver (data roots, mesh locations, etc.).
atek_variant ATEK payload variant and on-disk shard subdirectory under the data root.
scene_ids Optional list of ASE scene IDs to include.
snippet_ids Optional snippet IDs to include for each scene.
snippet_key_filter Optional sample key filter applied after loading shards.
tar_urls ATEK WebDataset shard paths or globs carrying tensorized ASE snippets.
scene_to_mesh Optional mapping of scene_id to ASE GT-mesh evaluation asset.
taxonomy_csv_filename Filename for the ATEK→EFM taxonomy mapping CSV.
batch_size WebDataset batch size inside ATEK loader.
wds_shuffle Enable shuffled sampling in the underlying WebDataset pipeline.
wds_repeat Repeat the underlying WebDataset stream (infinite iterator).
snippet_length_s Snippet duration in seconds used by the EFM adaptor.
freq_hz Sampling frequency (Hz) used by the EFM adaptor.
semidense_points_pad Pad/clip semi-dense points to this count per snippet frame.
load_meshes Attach GT meshes to snippets when available.
require_mesh Raise if a GT mesh is missing for any requested scene.
mesh_simplify_ratio Fraction of faces to keep when simplifying meshes. None disables.
crop_mesh If True, crop meshes to snippet bounds before simplification.
mesh_crop_margin_m Margin added to semidense bounds when cropping meshes. None disables cropping.
cache_meshes Cache loaded meshes in memory per dataset instance.
mesh_crop_min_keep_ratio Minimum face keep ratio tolerated during mesh cropping.
device Device to move snippet tensors onto.
verbosity Verbosity level for dataset logging.
is_debug Enable debug logging and extra diagnostics.
taxonomy_csv Resolve the ATEK-to-EFM semantic taxonomy used by EfmModelAdaptor.

1.4.2.2 Methods

Name Description
setup_target Instantiate the configured raw EFM dataset.
1.4.2.2.1 setup_target
data_handling.ase_efm.AseEfmDatasetConfig.setup_target()

Instantiate the configured raw EFM dataset.

1.4.3 EfmCameraView

data_handling.ase_efm.EfmCameraView(
    images,
    calib,
    time_ns,
    frame_ids,
    distance_m=None,
    distance_time_ns=None,
)

Expose one calibrated Aria camera stream view without copying EFM tensors.

The view consumes matching entries from ARIA_IMG, ARIA_CALIB, ARIA_IMG_TIME_NS, ARIA_FRAME_ID, and optional ARIA_DISTANCE_M/ARIA_DEPTH_TIME_NS families. VRS and online calibration are upstream provenance; runtime access is through ATEK/EFM.

1.4.3.1 Attributes

Name Description
images Tensor["F C H W", float32] normalized RGB or monochrome frames.
calib Per-frame CameraTW intrinsics/extrinsics; backing shape is (F, 34).
time_ns Tensor["F", int64] Aria device-clock timestamps aligned to images.
frame_ids Tensor["F", int64\|float32] frame ids within the snippet.
distance_m Optional Tensor["F 1 H W", float32] per-pixel ray distances in metres.
distance_time_ns Optional Tensor["F", int64] timestamps for distance_m.
num_frames Return the number of frames in the camera stream.

1.4.3.2 Methods

Name Description
to Move the camera view tensors to the requested device and dtype.
get_fov Return Tensor["F 2", float32] horizontal/vertical FOV in degrees.
select_frame_indices Resolve user-provided frame indices, supporting negatives and defaults.
nearest_traj_indices Return selected camera indices and nearest trajectory indices.
1.4.3.2.1 to
data_handling.ase_efm.EfmCameraView.to(device, *, dtype=None)

Move the camera view tensors to the requested device and dtype.

1.4.3.2.2 get_fov
data_handling.ase_efm.EfmCameraView.get_fov()

Return Tensor["F 2", float32] horizontal/vertical FOV in degrees.

1.4.3.2.3 select_frame_indices
data_handling.ase_efm.EfmCameraView.select_frame_indices(
    frame_indices,
    *,
    default_last,
)

Resolve user-provided frame indices, supporting negatives and defaults.

1.4.3.2.4 nearest_traj_indices
data_handling.ase_efm.EfmCameraView.nearest_traj_indices(
    traj_ts_ns,
    frame_indices=None,
    *,
    default_last=True,
)

Return selected camera indices and nearest trajectory indices.

1.4.4 EfmGTView

data_handling.ase_efm.EfmGTView(raw)

Parse nested gt_data/efm_gt annotations for evaluation-only access.

1.4.4.1 Attributes

Name Description
raw Zero-copy backing GT payload from the adapted ATEK sample.
efm_gt Timestamp-indexed per-camera GT views; never actor-visible observations.
timestamps Return sorted string keys for available efm_gt timestamps.

1.4.4.2 Methods

Name Description
cameras_at Return evaluation-only per-camera GT OBBs at one timestamp.
1.4.4.2.1 cameras_at
data_handling.ase_efm.EfmGTView.cameras_at(ts)

Return evaluation-only per-camera GT OBBs at one timestamp.

1.4.4.2.1.1 Parameters
Name Type Description Default
ts str | int Timestamp key accepted as text or an integer. required
1.4.4.2.1.2 Returns
Name Type Description
dict[str, EfmGtCameraObbView] Camera-name mapping for the requested GT slice.

1.4.5 EfmObbView

data_handling.ase_efm.EfmObbView(obbs, hz=None)

Expose snippet-level GT boxes in the EFM ObbTW layout.

obbs may be shaped (T, K, 34), (1, K, 34), or (K, 34) depending on the source. K is padded; padded rows are all PAD_VAL and must be removed with ObbTW.get_padding_mask() before matching label targets.

The 34 payload columns are the EFM OBB contract: object-frame 3D bounds, three 2D boxes, a 12-value pose, semantic id, instance id, confidence, and movable flag. In EfmSnippetView these boxes originate from ARIA_OBB_PADDED and are GT label/evaluation data. Actor-visible target hypotheses must come from EVL predictions or tracking instead. Downstream target selection transforms the latest valid slice into world coordinates before constructing Oracle target tasks.

1.4.5.1 Attributes

Name Description
obbs Tensor["... K 34", float32] GT boxes in the source EFM frame.
hz Optional Tensor["1", int32] detection frequency.

1.4.6 EfmPointsView

data_handling.ase_efm.EfmPointsView(
    points_world,
    dist_std,
    inv_dist_std,
    time_ns,
    volume_min,
    volume_max,
    lengths,
)

Expose actor-visible MPS semidense reconstruction evidence.

The ARIA_POINTS_* tensors are padded per frame; lengths defines the valid prefix and non-finite rows remain invalid. Positions, uncertainties, and volume bounds preserve the EFM world frame and metric units.

1.4.6.1 Attributes

Name Description
points_world Tensor["F N_pad 3", float32] padded world-frame MPS points in metres.
dist_std Tensor["F N_pad", float32] per-point distance standard deviation in metres.
inv_dist_std Tensor["F N_pad", float32] inverse distance uncertainty in metres⁻¹.
time_ns Tensor["F", int64] timestamps aligned to point slices.
volume_min Tensor["3", float32] snippet AABB minimum in world frame, metres.
volume_max Tensor["3", float32] snippet AABB maximum in world frame, metres.
lengths Tensor["F", int64] valid point-prefix length for each padded frame.

1.4.6.2 Methods

Name Description
to Move the semidense point tensors to the requested device and dtype.
collapse_points Collapse points across time and optionally subsample.
last_frame_points_np Return points from the latest frame with valid points, subsampled if needed.
1.4.6.2.1 to
data_handling.ase_efm.EfmPointsView.to(device, *, dtype=None)

Move the semidense point tensors to the requested device and dtype.

1.4.6.2.2 collapse_points
data_handling.ase_efm.EfmPointsView.collapse_points(
    max_points=None,
    include_inv_dist_std=False,
    include_obs_count=False,
)

Collapse points across time and optionally subsample.

1.4.6.2.2.1 Parameters
Name Type Description Default
max_points int | None Optional cap on the number of returned points. None
include_inv_dist_std bool If True, append inverse depth std per point. False
include_obs_count bool If True, append the number of snippet frames that observed each point (observation count). False
1.4.6.2.2.2 Returns
Name Type Description
Tensor Tensor["K 3", float32] if no extra features are requested.
Tensor Tensor["K 4", float32] if exactly one extra feature is requested
Tensor (XYZ + inv_dist_std or XYZ + obs_count).
Tensor Tensor["K 5", float32] if both extras are requested
Tensor (XYZ + inv_dist_std + obs_count).
1.4.6.2.3 last_frame_points_np
data_handling.ase_efm.EfmPointsView.last_frame_points_np(max_points=None)

Return points from the latest frame with valid points, subsampled if needed.

1.4.7 EfmSnippetLoader

data_handling.ase_efm.EfmSnippetLoader(
    dataset_payload,
    device,
    paths,
    include_gt_mesh,
)

Resolve source snippets through worker-owned, per-scene dataset caches.

Instances are intentionally not a persisted store component. They reopen ATEK WebDataset shards when a reader requests raw camera, MPS, calibration, or optional GT-mesh context that was not duplicated into offline blocks.

1.4.7.1 Methods

Name Description
load Load one source snippet, rebuilding a stale scene iterator once.
1.4.7.1.1 load
data_handling.ase_efm.EfmSnippetLoader.load(scene_id, snippet_id)

Load one source snippet, rebuilding a stale scene iterator once.

1.4.7.1.1.1 Parameters
Name Type Description Default
scene_id str ASE scene identifier used to locate ATEK shards. required
snippet_id str Compact or raw ATEK sample identifier to match. required
1.4.7.1.1.2 Returns
Name Type Description
Live EfmSnippetView class:EfmSnippetView backed by the source shard.

1.4.8 EfmSnippetView

data_handling.ase_efm.EfmSnippetView(
    efm,
    scene_id,
    snippet_id,
    mesh=None,
    crop_bounds=None,
    mesh_verts=None,
    mesh_faces=None,
    mesh_cache_key=None,
    mesh_specs=None,
)

Own the typed boundary around one adapted ATEK/ASE snippet.

efm retains the zero-copy key/value payload produced by EfmModelAdaptor. Camera/calibration, world-from-rig trajectory, and MPS semidense fields are actor-visible observations. Attached ASE meshes, ARIA_OBB_PADDED, and nested gt_data are oracle/evaluation assets. Mesh tensors are optional CPU/cache products and are not moved into the EFM dictionary.

1.4.8.1 Attributes

Name Description
efm Zero-copy adapted ATEK payload containing the consumed ARIA_* key families.
scene_id ASE scene identifier used for shard lineage and GT-mesh pairing.
snippet_id Compact ATEK sample identifier derived from the WebDataset sample key.
mesh Optional ASE GT mesh used only by oracle labeling and evaluation.
crop_bounds Optional pair of Tensor["3", float32] world-frame bounds in metres.
mesh_verts Optional Tensor["V 3", float32] GT-mesh vertices in world frame, metres.
mesh_faces Optional Tensor["F_mesh 3", int64] triangle vertex indices.
mesh_cache_key Stable key (spec hash) for shared mesh caches across components.
mesh_specs Optional mesh processing specification used to derive attached mesh tensors.
camera_rgb RGB stream (fisheye624, 240x240 in ASE preprocessing).
camera_slam_left Left SLAM mono stream (fisheye624, ~320x240).
camera_slam_right Right SLAM mono stream (fisheye624, ~320x240).
trajectory Rig trajectory (world←rig) aligned to snippet frames.
semidense Semi-dense SLAM points (padded to fixed length).
obbs Return padded ASE GT OBBs for label/evaluation use when present.
gt Ground-truth dict gt_data (EFM OBBs per timestamp/camera).
has_mesh Return whether a ground-truth mesh is attached to the snippet.

1.4.8.2 Methods

Name Description
from_cache_efm Construct a snippet view from an offline-cache EFM dict.
get_camera Return the requested camera stream view from the backing EFM dict.
get_occupancy_extend Return [xmin, xmax, ymin, ymax, zmin, zmax] AABB in world frame.
to Return a shallow copy with EFM tensors and wrappers moved to a device.
prune_efm Return a view with the EFM dict pruned to the requested keys.
1.4.8.2.1 from_cache_efm
data_handling.ase_efm.EfmSnippetView.from_cache_efm(
    efm,
    *,
    mesh=None,
    crop_bounds=None,
    mesh_verts=None,
    mesh_faces=None,
    mesh_cache_key=None,
    mesh_specs=None,
)

Construct a snippet view from an offline-cache EFM dict.

1.4.8.2.1.1 Parameters
Name Type Description Default
efm dict[str, Any] Raw cache sample dict that includes "__key__". required
mesh Trimesh | None Optional GT mesh to attach. None
crop_bounds tuple[torch.Tensor, torch.Tensor] | None Optional world-space AABB override. None
mesh_verts torch.Tensor | None Optional cached mesh vertices. None
mesh_faces torch.Tensor | None Optional cached mesh faces. None
mesh_cache_key str | None Optional mesh cache key for shared caches. None
mesh_specs MeshProcessSpec | None Optional mesh processing spec. None
1.4.8.2.1.2 Returns
Name Type Description
'EfmSnippetView' Parsed EfmSnippetView instance.
1.4.8.2.2 get_camera
data_handling.ase_efm.EfmSnippetView.get_camera(prefix)

Return the requested camera stream view from the backing EFM dict.

1.4.8.2.3 get_occupancy_extend
data_handling.ase_efm.EfmSnippetView.get_occupancy_extend()

Return [xmin, xmax, ymin, ymax, zmin, zmax] AABB in world frame.

1.4.8.2.4 to
data_handling.ase_efm.EfmSnippetView.to(device, *, dtype=None)

Return a shallow copy with EFM tensors and wrappers moved to a device.

The backing dictionary is rebuilt without mutating this view. Plain tensors honor dtype; EFM wrappers preserve their own dtype contract. The CPU Trimesh object is shared, while cached mesh tensors and world-frame crop bounds are moved.

1.4.8.2.5 prune_efm
data_handling.ase_efm.EfmSnippetView.prune_efm(keep_keys, *, keep_prefixes=())

Return a view with the EFM dict pruned to the requested keys.

1.4.8.2.5.1 Parameters
Name Type Description Default
keep_keys set[str] | None Exact EFM keys to retain. None returns self unchanged. required
keep_prefixes tuple[str, …] Optional prefixes; keys starting with any prefix are retained. ()

1.4.9 EfmTrajectoryView

data_handling.ase_efm.EfmTrajectoryView(t_world_rig, time_ns, gravity_in_world)

Expose the MPS/EFM world-from-rig trajectory for one snippet.

1.4.9.1 Attributes

Name Description
t_world_rig PoseTW["F 12"] world←rig transforms; translation is metres.
time_ns Tensor["F", int64] pose timestamps.
gravity_in_world Tensor["3", float32] world-frame gravity in metres per second squared.
final_pose Return the final PoseTW["12"] world←rig pose in the snippet.

1.4.9.2 Methods

Name Description
to Move the trajectory tensors to the requested device and dtype.
1.4.9.2.1 to
data_handling.ase_efm.EfmTrajectoryView.to(device, *, dtype=None)

Move the trajectory tensors to the requested device and dtype.

1.5 Functions

Name Description
infer_semidense_bounds Infer snippet world-space AABB from semidense metadata or points.
is_efm_snippet_view_instance Return whether value behaves like an EfmSnippetView.

1.5.1 infer_semidense_bounds

data_handling.ase_efm.infer_semidense_bounds(efm_dict)

Infer snippet world-space AABB from semidense metadata or points.

Metadata bounds and finite-point bounds are compared when both exist; the smaller finite volume is retained to avoid an unnecessarily loose crop. Padded and non-finite semidense rows never contribute.

1.5.1.1 Parameters

Name Type Description Default
efm_dict Mapping[str, Any] Adapted EFM payload containing world-frame semidense points and/or explicit volume bounds. required

1.5.1.2 Returns

Name Type Description
tuple[torch.Tensor, torch.Tensor] | None Pair of Tensor["3", float32] world-frame AABB minima and maxima in
tuple[torch.Tensor, torch.Tensor] | None metres on CPU, or None when no finite bounds are available.

1.5.2 is_efm_snippet_view_instance

data_handling.ase_efm.is_efm_snippet_view_instance(value)

Return whether value behaves like an EfmSnippetView.

The v2 stack accepts both the local data-handling view classes and legacy view objects that expose the same public attributes.

Back to top