1 data_handling.ase_efm
data_handling.ase_efm
Raw ASE/EFM data interfaces used by aria_nbv.data_handling.
This module provides the stable raw snippet layer for the data-handling package. It keeps the existing raw dataset and typed view implementations intact while presenting a single import surface for:
- the iterable raw ASE/EFM dataset,
- typed raw and VIN snippet views, and
- the worker-local snippet loader used for live snippet attachment.
1.1 Package guide
aria_nbv.data_handling.ase_efm streams tensorized ASE snippets from ATEK WebDataset shards and presents them as typed EFM views. It is the live input used by candidate generation, immutable VIN-store construction, and small rollout-generation smoke tests.
ATEK shards carry calibrated observations, trajectory, and MPS semidense points. Optional ASE meshes and GT annotations are attached as oracle/evaluation assets and must not become ordinary actor inputs.
1.2 Read Snippets
from aria_nbv.data_handling.ase_efm import AseEfmDatasetConfig
dataset = AseEfmDatasetConfig(
scene_ids=["81286"],
load_meshes=True,
require_mesh=True,
device="cpu",
).setup_target()
snippet = next(iter(dataset))
print(snippet.scene_id, snippet.snippet_id)AseEfmDatasetConfig resolves scene, shard, and sample filters; applies the ATEK-to-EFM taxonomy; and optionally pairs ASE meshes. EfmSnippetView exposes typed camera, trajectory, semidense-point, OBB, and GT surfaces without adding a parallel field schema. EfmSnippetLoader is the worker-local adapter for reattaching a known raw snippet to an immutable store row.
Use the downloader before constructing the dataset:
cd aria_nbv
uv run nbv-downloader -m list -c efm -n 10
uv run nbv-downloader -m download -c efm --ns 1 --max-shards 1Detailed field, frame, and padding contracts live in the module docstrings and generated API reference.
1.3 Verification
cd aria_nbv
uv run ruff check aria_nbv/data_handling/ase_efm
uv run pytest tests/data_handling/test_efm_dataset_snippet.py \
tests/data_handling/test_efm_snippet_loader.py1.4 Classes
| Name | Description |
|---|---|
| AseEfmDataset | Stream zero-copy EFM views from ATEK shards with optional ASE meshes. |
| AseEfmDatasetConfig | Configure the canonical ATEK-shard to EFM-snippet streaming path. |
| EfmCameraView | Expose one calibrated Aria camera stream view without copying EFM tensors. |
| EfmGTView | Parse nested gt_data/efm_gt annotations for evaluation-only access. |
| EfmObbView | Expose snippet-level GT boxes in the EFM ObbTW layout. |
| EfmPointsView | Expose actor-visible MPS semidense reconstruction evidence. |
| EfmSnippetLoader | Resolve source snippets through worker-owned, per-scene dataset caches. |
| EfmSnippetView | Own the typed boundary around one adapted ATEK/ASE snippet. |
| EfmTrajectoryView | Expose the MPS/EFM world-from-rig trajectory for one snippet. |
1.4.1 AseEfmDataset
data_handling.ase_efm.AseEfmDataset(config)Stream zero-copy EFM views from ATEK shards with optional ASE meshes.
Each iteration adapts one WebDataset sample into the consumed EFM key families, infers stable scene/snippet lineage, and attaches processed mesh tensors when configured. The iterable owns per-instance mesh memoization; yielded EfmSnippetView objects retain the backing EFM dictionary. Meshes never become actor-visible observations.
1.4.2 AseEfmDatasetConfig
data_handling.ase_efm.AseEfmDatasetConfig()Configure the canonical ATEK-shard to EFM-snippet streaming path.
The config resolves scene/shard/sample filters, the ATEK-to-EFM taxonomy, and optional ASE mesh pairing before constructing AseEfmDataset. EfmModelAdaptor owns flattened-key remapping; this config does not introduce a parallel schema.
1.4.2.1 Attributes
| Name | Description |
|---|---|
| cache_exclude_fields | Fields omitted from cache snapshots because they are large or derived. |
| target_type | Factory target for BaseConfig.setup_target. |
| paths | Project path resolver (data roots, mesh locations, etc.). |
| atek_variant | ATEK payload variant and on-disk shard subdirectory under the data root. |
| scene_ids | Optional list of ASE scene IDs to include. |
| snippet_ids | Optional snippet IDs to include for each scene. |
| snippet_key_filter | Optional sample key filter applied after loading shards. |
| tar_urls | ATEK WebDataset shard paths or globs carrying tensorized ASE snippets. |
| scene_to_mesh | Optional mapping of scene_id to ASE GT-mesh evaluation asset. |
| taxonomy_csv_filename | Filename for the ATEK→EFM taxonomy mapping CSV. |
| batch_size | WebDataset batch size inside ATEK loader. |
| wds_shuffle | Enable shuffled sampling in the underlying WebDataset pipeline. |
| wds_repeat | Repeat the underlying WebDataset stream (infinite iterator). |
| snippet_length_s | Snippet duration in seconds used by the EFM adaptor. |
| freq_hz | Sampling frequency (Hz) used by the EFM adaptor. |
| semidense_points_pad | Pad/clip semi-dense points to this count per snippet frame. |
| load_meshes | Attach GT meshes to snippets when available. |
| require_mesh | Raise if a GT mesh is missing for any requested scene. |
| mesh_simplify_ratio | Fraction of faces to keep when simplifying meshes. None disables. |
| crop_mesh | If True, crop meshes to snippet bounds before simplification. |
| mesh_crop_margin_m | Margin added to semidense bounds when cropping meshes. None disables cropping. |
| cache_meshes | Cache loaded meshes in memory per dataset instance. |
| mesh_crop_min_keep_ratio | Minimum face keep ratio tolerated during mesh cropping. |
| device | Device to move snippet tensors onto. |
| verbosity | Verbosity level for dataset logging. |
| is_debug | Enable debug logging and extra diagnostics. |
| taxonomy_csv | Resolve the ATEK-to-EFM semantic taxonomy used by EfmModelAdaptor. |
1.4.2.2 Methods
| Name | Description |
|---|---|
| setup_target | Instantiate the configured raw EFM dataset. |
1.4.2.2.1 setup_target
data_handling.ase_efm.AseEfmDatasetConfig.setup_target()Instantiate the configured raw EFM dataset.
1.4.3 EfmCameraView
data_handling.ase_efm.EfmCameraView(
images,
calib,
time_ns,
frame_ids,
distance_m=None,
distance_time_ns=None,
)Expose one calibrated Aria camera stream view without copying EFM tensors.
The view consumes matching entries from ARIA_IMG, ARIA_CALIB, ARIA_IMG_TIME_NS, ARIA_FRAME_ID, and optional ARIA_DISTANCE_M/ARIA_DEPTH_TIME_NS families. VRS and online calibration are upstream provenance; runtime access is through ATEK/EFM.
1.4.3.1 Attributes
| Name | Description |
|---|---|
| images | Tensor["F C H W", float32] normalized RGB or monochrome frames. |
| calib | Per-frame CameraTW intrinsics/extrinsics; backing shape is (F, 34). |
| time_ns | Tensor["F", int64] Aria device-clock timestamps aligned to images. |
| frame_ids | Tensor["F", int64\|float32] frame ids within the snippet. |
| distance_m | Optional Tensor["F 1 H W", float32] per-pixel ray distances in metres. |
| distance_time_ns | Optional Tensor["F", int64] timestamps for distance_m. |
| num_frames | Return the number of frames in the camera stream. |
1.4.3.2 Methods
| Name | Description |
|---|---|
| to | Move the camera view tensors to the requested device and dtype. |
| get_fov | Return Tensor["F 2", float32] horizontal/vertical FOV in degrees. |
| select_frame_indices | Resolve user-provided frame indices, supporting negatives and defaults. |
| nearest_traj_indices | Return selected camera indices and nearest trajectory indices. |
1.4.3.2.1 to
data_handling.ase_efm.EfmCameraView.to(device, *, dtype=None)Move the camera view tensors to the requested device and dtype.
1.4.3.2.2 get_fov
data_handling.ase_efm.EfmCameraView.get_fov()Return Tensor["F 2", float32] horizontal/vertical FOV in degrees.
1.4.3.2.3 select_frame_indices
data_handling.ase_efm.EfmCameraView.select_frame_indices(
frame_indices,
*,
default_last,
)Resolve user-provided frame indices, supporting negatives and defaults.
1.4.3.2.4 nearest_traj_indices
data_handling.ase_efm.EfmCameraView.nearest_traj_indices(
traj_ts_ns,
frame_indices=None,
*,
default_last=True,
)Return selected camera indices and nearest trajectory indices.
1.4.4 EfmGTView
data_handling.ase_efm.EfmGTView(raw)Parse nested gt_data/efm_gt annotations for evaluation-only access.
1.4.4.1 Attributes
| Name | Description |
|---|---|
| raw | Zero-copy backing GT payload from the adapted ATEK sample. |
| efm_gt | Timestamp-indexed per-camera GT views; never actor-visible observations. |
| timestamps | Return sorted string keys for available efm_gt timestamps. |
1.4.4.2 Methods
| Name | Description |
|---|---|
| cameras_at | Return evaluation-only per-camera GT OBBs at one timestamp. |
1.4.4.2.1 cameras_at
data_handling.ase_efm.EfmGTView.cameras_at(ts)Return evaluation-only per-camera GT OBBs at one timestamp.
1.4.4.2.1.1 Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| ts | str | int | Timestamp key accepted as text or an integer. | required |
1.4.4.2.1.2 Returns
| Name | Type | Description |
|---|---|---|
| dict[str, EfmGtCameraObbView] | Camera-name mapping for the requested GT slice. |
1.4.5 EfmObbView
data_handling.ase_efm.EfmObbView(obbs, hz=None)Expose snippet-level GT boxes in the EFM ObbTW layout.
obbs may be shaped (T, K, 34), (1, K, 34), or (K, 34) depending on the source. K is padded; padded rows are all PAD_VAL and must be removed with ObbTW.get_padding_mask() before matching label targets.
The 34 payload columns are the EFM OBB contract: object-frame 3D bounds, three 2D boxes, a 12-value pose, semantic id, instance id, confidence, and movable flag. In EfmSnippetView these boxes originate from ARIA_OBB_PADDED and are GT label/evaluation data. Actor-visible target hypotheses must come from EVL predictions or tracking instead. Downstream target selection transforms the latest valid slice into world coordinates before constructing Oracle target tasks.
1.4.5.1 Attributes
| Name | Description |
|---|---|
| obbs | Tensor["... K 34", float32] GT boxes in the source EFM frame. |
| hz | Optional Tensor["1", int32] detection frequency. |
1.4.6 EfmPointsView
data_handling.ase_efm.EfmPointsView(
points_world,
dist_std,
inv_dist_std,
time_ns,
volume_min,
volume_max,
lengths,
)Expose actor-visible MPS semidense reconstruction evidence.
The ARIA_POINTS_* tensors are padded per frame; lengths defines the valid prefix and non-finite rows remain invalid. Positions, uncertainties, and volume bounds preserve the EFM world frame and metric units.
1.4.6.1 Attributes
| Name | Description |
|---|---|
| points_world | Tensor["F N_pad 3", float32] padded world-frame MPS points in metres. |
| dist_std | Tensor["F N_pad", float32] per-point distance standard deviation in metres. |
| inv_dist_std | Tensor["F N_pad", float32] inverse distance uncertainty in metres⁻¹. |
| time_ns | Tensor["F", int64] timestamps aligned to point slices. |
| volume_min | Tensor["3", float32] snippet AABB minimum in world frame, metres. |
| volume_max | Tensor["3", float32] snippet AABB maximum in world frame, metres. |
| lengths | Tensor["F", int64] valid point-prefix length for each padded frame. |
1.4.6.2 Methods
| Name | Description |
|---|---|
| to | Move the semidense point tensors to the requested device and dtype. |
| collapse_points | Collapse points across time and optionally subsample. |
| last_frame_points_np | Return points from the latest frame with valid points, subsampled if needed. |
1.4.6.2.1 to
data_handling.ase_efm.EfmPointsView.to(device, *, dtype=None)Move the semidense point tensors to the requested device and dtype.
1.4.6.2.2 collapse_points
data_handling.ase_efm.EfmPointsView.collapse_points(
max_points=None,
include_inv_dist_std=False,
include_obs_count=False,
)Collapse points across time and optionally subsample.
1.4.6.2.2.1 Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| max_points | int | None | Optional cap on the number of returned points. | None |
| include_inv_dist_std | bool | If True, append inverse depth std per point. | False |
| include_obs_count | bool | If True, append the number of snippet frames that observed each point (observation count). | False |
1.4.6.2.2.2 Returns
| Name | Type | Description |
|---|---|---|
| Tensor | Tensor["K 3", float32] if no extra features are requested. |
|
| Tensor | Tensor["K 4", float32] if exactly one extra feature is requested |
|
| Tensor | (XYZ + inv_dist_std or XYZ + obs_count). | |
| Tensor | Tensor["K 5", float32] if both extras are requested |
|
| Tensor | (XYZ + inv_dist_std + obs_count). |
1.4.6.2.3 last_frame_points_np
data_handling.ase_efm.EfmPointsView.last_frame_points_np(max_points=None)Return points from the latest frame with valid points, subsampled if needed.
1.4.7 EfmSnippetLoader
data_handling.ase_efm.EfmSnippetLoader(
dataset_payload,
device,
paths,
include_gt_mesh,
)Resolve source snippets through worker-owned, per-scene dataset caches.
Instances are intentionally not a persisted store component. They reopen ATEK WebDataset shards when a reader requests raw camera, MPS, calibration, or optional GT-mesh context that was not duplicated into offline blocks.
1.4.7.1 Methods
| Name | Description |
|---|---|
| load | Load one source snippet, rebuilding a stale scene iterator once. |
1.4.7.1.1 load
data_handling.ase_efm.EfmSnippetLoader.load(scene_id, snippet_id)Load one source snippet, rebuilding a stale scene iterator once.
1.4.7.1.1.1 Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| scene_id | str | ASE scene identifier used to locate ATEK shards. | required |
| snippet_id | str | Compact or raw ATEK sample identifier to match. | required |
1.4.7.1.1.2 Returns
| Name | Type | Description |
|---|---|---|
| Live | EfmSnippetView | class:EfmSnippetView backed by the source shard. |
1.4.8 EfmSnippetView
data_handling.ase_efm.EfmSnippetView(
efm,
scene_id,
snippet_id,
mesh=None,
crop_bounds=None,
mesh_verts=None,
mesh_faces=None,
mesh_cache_key=None,
mesh_specs=None,
)Own the typed boundary around one adapted ATEK/ASE snippet.
efm retains the zero-copy key/value payload produced by EfmModelAdaptor. Camera/calibration, world-from-rig trajectory, and MPS semidense fields are actor-visible observations. Attached ASE meshes, ARIA_OBB_PADDED, and nested gt_data are oracle/evaluation assets. Mesh tensors are optional CPU/cache products and are not moved into the EFM dictionary.
1.4.8.1 Attributes
| Name | Description |
|---|---|
| efm | Zero-copy adapted ATEK payload containing the consumed ARIA_* key families. |
| scene_id | ASE scene identifier used for shard lineage and GT-mesh pairing. |
| snippet_id | Compact ATEK sample identifier derived from the WebDataset sample key. |
| mesh | Optional ASE GT mesh used only by oracle labeling and evaluation. |
| crop_bounds | Optional pair of Tensor["3", float32] world-frame bounds in metres. |
| mesh_verts | Optional Tensor["V 3", float32] GT-mesh vertices in world frame, metres. |
| mesh_faces | Optional Tensor["F_mesh 3", int64] triangle vertex indices. |
| mesh_cache_key | Stable key (spec hash) for shared mesh caches across components. |
| mesh_specs | Optional mesh processing specification used to derive attached mesh tensors. |
| camera_rgb | RGB stream (fisheye624, 240x240 in ASE preprocessing). |
| camera_slam_left | Left SLAM mono stream (fisheye624, ~320x240). |
| camera_slam_right | Right SLAM mono stream (fisheye624, ~320x240). |
| trajectory | Rig trajectory (world←rig) aligned to snippet frames. |
| semidense | Semi-dense SLAM points (padded to fixed length). |
| obbs | Return padded ASE GT OBBs for label/evaluation use when present. |
| gt | Ground-truth dict gt_data (EFM OBBs per timestamp/camera). |
| has_mesh | Return whether a ground-truth mesh is attached to the snippet. |
1.4.8.2 Methods
| Name | Description |
|---|---|
| from_cache_efm | Construct a snippet view from an offline-cache EFM dict. |
| get_camera | Return the requested camera stream view from the backing EFM dict. |
| get_occupancy_extend | Return [xmin, xmax, ymin, ymax, zmin, zmax] AABB in world frame. |
| to | Return a shallow copy with EFM tensors and wrappers moved to a device. |
| prune_efm | Return a view with the EFM dict pruned to the requested keys. |
1.4.8.2.1 from_cache_efm
data_handling.ase_efm.EfmSnippetView.from_cache_efm(
efm,
*,
mesh=None,
crop_bounds=None,
mesh_verts=None,
mesh_faces=None,
mesh_cache_key=None,
mesh_specs=None,
)Construct a snippet view from an offline-cache EFM dict.
1.4.8.2.1.1 Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| efm | dict[str, Any] | Raw cache sample dict that includes "__key__". |
required |
| mesh | Trimesh | None |
Optional GT mesh to attach. | None |
| crop_bounds | tuple[torch.Tensor, torch.Tensor] | None | Optional world-space AABB override. | None |
| mesh_verts | torch.Tensor | None | Optional cached mesh vertices. | None |
| mesh_faces | torch.Tensor | None | Optional cached mesh faces. | None |
| mesh_cache_key | str | None | Optional mesh cache key for shared caches. | None |
| mesh_specs | MeshProcessSpec | None | Optional mesh processing spec. | None |
1.4.8.2.1.2 Returns
| Name | Type | Description |
|---|---|---|
| 'EfmSnippetView' | Parsed EfmSnippetView instance. |
1.4.8.2.2 get_camera
data_handling.ase_efm.EfmSnippetView.get_camera(prefix)Return the requested camera stream view from the backing EFM dict.
1.4.8.2.3 get_occupancy_extend
data_handling.ase_efm.EfmSnippetView.get_occupancy_extend()Return [xmin, xmax, ymin, ymax, zmin, zmax] AABB in world frame.
1.4.8.2.4 to
data_handling.ase_efm.EfmSnippetView.to(device, *, dtype=None)Return a shallow copy with EFM tensors and wrappers moved to a device.
The backing dictionary is rebuilt without mutating this view. Plain tensors honor dtype; EFM wrappers preserve their own dtype contract. The CPU Trimesh object is shared, while cached mesh tensors and world-frame crop bounds are moved.
1.4.8.2.5 prune_efm
data_handling.ase_efm.EfmSnippetView.prune_efm(keep_keys, *, keep_prefixes=())Return a view with the EFM dict pruned to the requested keys.
1.4.8.2.5.1 Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| keep_keys | set[str] | None | Exact EFM keys to retain. None returns self unchanged. |
required |
| keep_prefixes | tuple[str, …] | Optional prefixes; keys starting with any prefix are retained. | () |
1.4.9 EfmTrajectoryView
data_handling.ase_efm.EfmTrajectoryView(t_world_rig, time_ns, gravity_in_world)Expose the MPS/EFM world-from-rig trajectory for one snippet.
1.4.9.1 Attributes
| Name | Description |
|---|---|
| t_world_rig | PoseTW["F 12"] world←rig transforms; translation is metres. |
| time_ns | Tensor["F", int64] pose timestamps. |
| gravity_in_world | Tensor["3", float32] world-frame gravity in metres per second squared. |
| final_pose | Return the final PoseTW["12"] world←rig pose in the snippet. |
1.4.9.2 Methods
| Name | Description |
|---|---|
| to | Move the trajectory tensors to the requested device and dtype. |
1.4.9.2.1 to
data_handling.ase_efm.EfmTrajectoryView.to(device, *, dtype=None)Move the trajectory tensors to the requested device and dtype.
1.5 Functions
| Name | Description |
|---|---|
| infer_semidense_bounds | Infer snippet world-space AABB from semidense metadata or points. |
| is_efm_snippet_view_instance | Return whether value behaves like an EfmSnippetView. |
1.5.1 infer_semidense_bounds
data_handling.ase_efm.infer_semidense_bounds(efm_dict)Infer snippet world-space AABB from semidense metadata or points.
Metadata bounds and finite-point bounds are compared when both exist; the smaller finite volume is retained to avoid an unnecessarily loose crop. Padded and non-finite semidense rows never contribute.
1.5.1.1 Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| efm_dict | Mapping[str, Any] | Adapted EFM payload containing world-frame semidense points and/or explicit volume bounds. | required |
1.5.1.2 Returns
| Name | Type | Description |
|---|---|---|
| tuple[torch.Tensor, torch.Tensor] | None | Pair of Tensor["3", float32] world-frame AABB minima and maxima in |
|
| tuple[torch.Tensor, torch.Tensor] | None | metres on CPU, or None when no finite bounds are available. |
1.5.2 is_efm_snippet_view_instance
data_handling.ase_efm.is_efm_snippet_view_instance(value)Return whether value behaves like an EfmSnippetView.
The v2 stack accepts both the local data-handling view classes and legacy view objects that expose the same public attributes.