1 data_handling.ase_efm.dataset

data_handling.ase_efm.dataset

EFM-formatted ASE dataset wrapper.

AseEfmDataset streams ATEK/ASE shards through the EFM adaptor and yields typed EfmSnippetView objects. It is the live data source used by candidate generation, VIN offline-cache construction, and rollout-data smoke builds.

ATEK shards are the tensorized provenance for calibrated VRS streams, trajectories, online calibration, and MPS semidense points; this module does not open raw VRS recordings. Semidense points and calibrated poses are actor-visible observations, while attached ASE meshes and GT annotations are oracle label/evaluation assets.

Scene/snippet identity, mesh attachment, taxonomy mapping, and semidense bounds are part of the contract. Any downstream store that depends on oracle labels must keep scene-level splits and manifest hashes instead of relying on sample order alone.

1.1 Classes

Name	Description
AseEfmDataset	Stream zero-copy EFM views from ATEK shards with optional ASE meshes.
AseEfmDatasetConfig	Configure the canonical ATEK-shard to EFM-snippet streaming path.

1.1.1 AseEfmDataset

data_handling.ase_efm.dataset.AseEfmDataset(config)

Stream zero-copy EFM views from ATEK shards with optional ASE meshes.

Each iteration adapts one WebDataset sample into the consumed EFM key families, infers stable scene/snippet lineage, and attaches processed mesh tensors when configured. The iterable owns per-instance mesh memoization; yielded EfmSnippetView objects retain the backing EFM dictionary. Meshes never become actor-visible observations.

1.1.2 AseEfmDatasetConfig

data_handling.ase_efm.dataset.AseEfmDatasetConfig()

Configure the canonical ATEK-shard to EFM-snippet streaming path.

The config resolves scene/shard/sample filters, the ATEK-to-EFM taxonomy, and optional ASE mesh pairing before constructing AseEfmDataset. EfmModelAdaptor owns flattened-key remapping; this config does not introduce a parallel schema.

1.1.2.1 Attributes

Name	Description
cache_exclude_fields	Fields omitted from cache snapshots because they are large or derived.
target_type	Factory target for BaseConfig.setup_target.
paths	Project path resolver (data roots, mesh locations, etc.).
atek_variant	ATEK payload variant and on-disk shard subdirectory under the data root.
scene_ids	Optional list of ASE scene IDs to include.
snippet_ids	Optional snippet IDs to include for each scene.
snippet_key_filter	Optional sample key filter applied after loading shards.
tar_urls	ATEK WebDataset shard paths or globs carrying tensorized ASE snippets.
scene_to_mesh	Optional mapping of scene_id to ASE GT-mesh evaluation asset.
taxonomy_csv_filename	Filename for the ATEK→EFM taxonomy mapping CSV.
batch_size	WebDataset batch size inside ATEK loader.
wds_shuffle	Enable shuffled sampling in the underlying WebDataset pipeline.
wds_repeat	Repeat the underlying WebDataset stream (infinite iterator).
snippet_length_s	Snippet duration in seconds used by the EFM adaptor.
freq_hz	Sampling frequency (Hz) used by the EFM adaptor.
semidense_points_pad	Pad/clip semi-dense points to this count per snippet frame.
load_meshes	Attach GT meshes to snippets when available.
require_mesh	Raise if a GT mesh is missing for any requested scene.
mesh_simplify_ratio	Fraction of faces to keep when simplifying meshes. None disables.
crop_mesh	If True, crop meshes to snippet bounds before simplification.
mesh_crop_margin_m	Margin added to semidense bounds when cropping meshes. None disables cropping.
cache_meshes	Cache loaded meshes in memory per dataset instance.
mesh_crop_min_keep_ratio	Minimum face keep ratio tolerated during mesh cropping.
device	Device to move snippet tensors onto.
verbosity	Verbosity level for dataset logging.
is_debug	Enable debug logging and extra diagnostics.
taxonomy_csv	Resolve the ATEK-to-EFM semantic taxonomy used by EfmModelAdaptor.

1.1.2.2 Methods

Name	Description
setup_target	Instantiate the configured raw EFM dataset.

1.1.2.2.1 setup_target

data_handling.ase_efm.dataset.AseEfmDatasetConfig.setup_target()

Instantiate the configured raw EFM dataset.