1 data_handling.atek_downloads

data_handling.atek_downloads

ASE mesh and ATEK shard acquisition utilities.

This package provides manifest parsing plus checksum-aware acquisition of ATEK WebDataset shards and ASE ground-truth meshes. Downloaded shards are the tensorized VRS/MPS provenance consumed by :class:AseEfmDataset; meshes remain oracle supervision and evaluation assets rather than actor-visible inputs.

1.1 Classes

Name Description
ASEDownloader Acquire manifest-selected ASE meshes and ATEK WebDataset shards.
ASEDownloaderConfig Configuration for ASE downloader with CLI support.
ASEMetadata Parse mesh + ATEK URL JSONs to expose scene-level metadata.
SceneMetadata Describe one ASE scene for a specific ATEK acquisition variant.

1.1.1 ASEDownloader

data_handling.atek_downloads.ASEDownloader(config)

Acquire manifest-selected ASE meshes and ATEK WebDataset shards.

The instance owns parsed ASEMetadata, download destinations, and console state for one run. Mesh ZIPs are SHA-1 verified and extracted into the ASE mesh cache; ATEK shard acquisition is delegated to the upstream downloader using a temporary filtered manifest that is removed afterward.

1.1.1.1 Methods

Name Description
download_scenes Download meshes + ATEK data for scenes.
download_meshes Download GT meshes for selected scenes after applying overwrite policy.
download_atek Download ATEK shards for selected ASE scenes.
download_scenes_with_meshes Download scenes that have meshes and at least min_shards shards for a given config.
1.1.1.1.1 download_scenes
data_handling.atek_downloads.ASEDownloader.download_scenes(
    scenes=None,
    scene_ids=None,
    download_meshes=True,
    download_atek=True,
)

Download meshes + ATEK data for scenes.

1.1.1.1.1.1 Parameters
Name Type Description Default
scenes list[SceneMetadata] | None Explicit scene records to acquire. None
scene_ids list[str] | None Additional scene IDs resolved through the parsed manifest. None
download_meshes bool Whether to acquire ASE GT meshes. True
download_atek bool Whether to acquire ATEK WebDataset shards. True
1.1.1.1.2 download_meshes
data_handling.atek_downloads.ASEDownloader.download_meshes(
    scene_ids=None,
    overwrite=False,
)

Download GT meshes for selected scenes after applying overwrite policy.

1.1.1.1.2.1 Parameters
Name Type Description Default
scene_ids list[str] | None Optional ASE scene allowlist; None selects all scenes. None
overwrite bool Whether existing archive/download state may be replaced. False
1.1.1.1.3 download_atek
data_handling.atek_downloads.ASEDownloader.download_atek(scene_ids=None)

Download ATEK shards for selected ASE scenes.

1.1.1.1.3.1 Parameters
Name Type Description Default
scene_ids list[str] | None Optional ASE scene allowlist; None selects all scenes. None
1.1.1.1.4 download_scenes_with_meshes
data_handling.atek_downloads.ASEDownloader.download_scenes_with_meshes(
    min_shards,
    config,
    overwrite,
)

Download scenes that have meshes and at least min_shards shards for a given config.

1.1.2 ASEDownloaderConfig

data_handling.atek_downloads.ASEDownloaderConfig()

Configuration for ASE downloader with CLI support.

Supports two CLI modes (explicitly selected via positional mode): 1. Download mode: Download N scenes with meshes + ATEK shards 2. List mode: List available scenes

Example (CLI - Download): $ python -m aria_nbv.data_handling.atek_downloads.downloader download –n_scenes=5 –max_shards=2 $ python -m aria_nbv.data_handling.atek_downloads.downloader download –ns=10 –skip_meshes

Example (CLI - List): $ python -m aria_nbv.data_handling.atek_downloads.downloader list –n=10

1.1.2.1 Attributes

Name Description
target_type Return ASEDownloader for config-as-factory construction.
mode CLI operation: acquire selected assets or list manifest scene coverage.
list_n Optional maximum scene rows printed by list mode.
paths Project path resolver for URL manifests, ATEK shards, and ASE meshes.
output_dir Optional requested output root; concrete destinations are resolved from paths.
verbosity Structured console verbosity used by download and listing operations.
is_debug Enable debug logging (forces max verbosity).
prefer_scenes_with_max_shards Prefer scenes with maximum shards when limiting number of scenes.
mesh_json_filename Filename of the mesh download URLs JSON in url_dir. Relative to paths.url_dir.
atek_json_filename Filename of the ATEK download URLs JSON in url_dir. Relative to paths.url_dir.
atek_config_name ATEK configuration name.
n_scenes Number of scenes to download (0 = all available scenes).
max_shards Maximum shards per scene (None = all shards).
skip_meshes Skip downloading GT meshes (only download ATEK data).
skip_atek Skip downloading ATEK WDS data (only download meshes).
train_val_split_ratio Train-validation split ratio for ATEK data.
random_seed Random seed for shuffling ATEK data sequences.
download_wds_to_local Download WebDataset files to local storage (vs streaming URLs).
overwrite Always overwrite existing ATEK files (re-download even if present).

1.1.2.2 Methods

Name Description
from_cli Instantiate config using pydantic-settings CLI parser (explicit opt-in).
settings_customise_sources Order explicit init, CLI, environment, dotenv, and secret settings sources.
1.1.2.2.1 from_cli
data_handling.atek_downloads.ASEDownloaderConfig.from_cli()

Instantiate config using pydantic-settings CLI parser (explicit opt-in).

1.1.2.2.2 settings_customise_sources
data_handling.atek_downloads.ASEDownloaderConfig.settings_customise_sources(
    settings_cls,
    init_settings,
    env_settings,
    dotenv_settings,
    file_secret_settings,
    cli_settings=None,
)

Order explicit init, CLI, environment, dotenv, and secret settings sources.

1.1.2.2.2.1 Returns
Name Type Description
Pydantic-settings sources in decreasing precedence order.

1.1.3 ASEMetadata

data_handling.atek_downloads.ASEMetadata(
    url_dir,
    mesh_json_filename='ase_mesh_download_urls.json',
    atek_json_filename='AriaSyntheticEnvironment_ATEK_download_urls.json',
)

Parse mesh + ATEK URL JSONs to expose scene-level metadata.

1.1.3.1 Methods

Name Description
get_scenes_with_meshes Return scenes that have GT meshes.
filter_scenes Filter scene records by ATEK variant, shard count, and GT-mesh availability.
get_scenes Return scene records in stable ID order with an optional shard cap.
save Persist the aggregate scene inventory as compact JSON.
load Load an aggregate scene snapshot while retaining its manifest directory.
1.1.3.1.1 get_scenes_with_meshes
data_handling.atek_downloads.ASEMetadata.get_scenes_with_meshes(config=None)

Return scenes that have GT meshes.

1.1.3.1.1.1 Parameters
Name Type Description Default
config str | None Optional ATEK config name. If provided, scenes are returned for that specific ATEK config. None
1.1.3.1.1.2 Returns
Name Type Description
list[SceneMetadata] List of scenes that have GT meshes.
1.1.3.1.2 filter_scenes
data_handling.atek_downloads.ASEMetadata.filter_scenes(
    min_shards=0,
    require_mesh=False,
    config=None,
)

Filter scene records by ATEK variant, shard count, and GT-mesh availability.

1.1.3.1.2.1 Parameters
Name Type Description Default
min_shards int Minimum number of manifest shards required. 0
require_mesh bool Whether to keep only scenes with a GT mesh URL. False
config str | None Optional ATEK variant; None uses the aggregate scene view. None
1.1.3.1.2.2 Returns
Name Type Description
list[SceneMetadata] Matching scene records without mutating the parsed manifests.
1.1.3.1.3 get_scenes
data_handling.atek_downloads.ASEMetadata.get_scenes(n=None, max_shards=None)

Return scene records in stable ID order with an optional shard cap.

1.1.3.1.3.1 Parameters
Name Type Description Default
n int | None Optional maximum number of scenes. None
max_shards int | None Optional per-scene shard limit applied to returned copies. None
1.1.3.1.3.2 Returns
Name Type Description
list[SceneMetadata] Sorted scene records; the parser-owned records remain unchanged.
1.1.3.1.4 save
data_handling.atek_downloads.ASEMetadata.save(path)

Persist the aggregate scene inventory as compact JSON.

The snapshot records mesh-scene membership and aggregate SceneMetadata values; URL-manifest files remain the authority for per-config reconstruction.

1.1.3.1.5 load
data_handling.atek_downloads.ASEMetadata.load(path)

Load an aggregate scene snapshot while retaining its manifest directory.

1.1.3.1.5.1 Parameters
Name Type Description Default
path Path JSON snapshot produced by ASEMetadata.save. required
1.1.3.1.5.2 Returns
Name Type Description
'ASEMetadata' Parsed metadata with aggregate scene records replaced by the snapshot.

1.1.4 SceneMetadata

data_handling.atek_downloads.SceneMetadata(
    scene_id,
    has_gt_mesh,
    mesh_url,
    mesh_sha,
    shard_count,
    shard_ids,
    atek_config,
    total_frames,
)

Describe one ASE scene for a specific ATEK acquisition variant.

1.1.4.1 Attributes

Name Description
scene_id Stable ASE scene identifier used by shard directories and mesh pairing.
has_gt_mesh Whether the URL manifest exposes an ASE ground-truth mesh for evaluation.
mesh_url Optional CDN URL for the zipped GT mesh; not actor-visible sample data.
mesh_sha Optional expected SHA-1 digest used to verify the downloaded mesh archive.
shard_count Number of ATEK WebDataset shards selected for atek_config.
shard_ids Normalized shard identifiers with the manifest’s _tar suffix removed.
atek_config ATEK payload variant, such as efm or efm_eval.
total_frames Manifest frame count when available; zero for shard-only entries.
Back to top