Setup Instructions

1 Setup Instructions

This page mirrors the portable root setup guide in SETUP.md. Keep host-specific mounts and private machine notes out of public docs.

1.1 Clone and Environment

Clone the repository, initialize submodules, and build the package environment from aria_nbv/:

git clone https://github.com/JanDuchscherer104/ARIA-NBV.git
cd ARIA-NBV
git submodule update --init --recursive

cd aria_nbv
mamba env create -f environment.yml
mamba activate aria-nbv
uv sync --all-extras

environment.yml provides Python 3.11 plus a CUDA 12.1 build toolchain on Linux. If uv cannot resolve Python 3.11 automatically, pass a local interpreter:

cd aria_nbv
UV_PYTHON=/path/to/python3.11 uv sync --all-extras

Check the interpreter before diagnosing missing dependencies:

cd aria_nbv
uv run python --version
uv run python - <<'PY'
import torch
print("torch", torch.__version__)
print("cuda_available", torch.cuda.is_available())
print("cuda_version", torch.version.cuda)
PY

Torch CUDA availability is not enough for rollout/oracle rendering. PyTorch3D is a compiled extension and must pass its own CUDA rasterization smoke:

cd aria_nbv
uv run python - <<'PY'
import torch
from pytorch3d.renderer import FoVPerspectiveCameras, MeshRasterizer, RasterizationSettings
from pytorch3d.structures import Meshes

if not torch.cuda.is_available():
    raise SystemExit("Torch CUDA is unavailable.")

device = torch.device("cuda")
verts = torch.tensor([[-0.5, -0.5, 2.0], [0.5, -0.5, 2.0], [0.0, 0.5, 2.0]], device=device)
faces = torch.tensor([[0, 1, 2]], dtype=torch.int64, device=device)
mesh = Meshes(verts=[verts], faces=[faces])
rasterizer = MeshRasterizer(
    cameras=FoVPerspectiveCameras(device=device),
    raster_settings=RasterizationSettings(image_size=8, blur_radius=0.0, faces_per_pixel=1),
)
rasterizer(mesh)
print("pytorch3d_cuda_rasterization_ok")
PY

1.2 CUDA and CPU Expectations

CPU-only machines are suitable for docs work, config validation, lightweight tests, immutable-store reads, and rollout-builder dry-run validation. They are not the expected path for full ASE oracle generation, EVL backbone materialization, PyTorch3D depth rendering, or VIN training.

Linux GPU machines are the expected path for the thesis data loop. The package pins PyTorch CUDA 12.1 wheels on Linux, and several configs use device = "auto" so they can fall back to CPU for debugging. Treat CPU fallback as a diagnostic convenience rather than the performance target.

Use the aria-nbv mamba environment as the CUDA build-toolchain context and the repo .venv as the uv runtime. If PyTorch3D reports Not compiled with GPU support, rebuild it from the activated toolchain environment:

cd aria_nbv
mamba activate aria-nbv
export CUDA_HOME="$CONDA_PREFIX"
export FORCE_CUDA=1
export TORCH_CUDA_ARCH_LIST="8.6"
uv sync --all-extras --reinstall-package pytorch3d --no-build-isolation-package pytorch3d --no-cache

Optional xFormers can be installed on Linux when EFM3D attention utilities need it:

cd aria_nbv
uv sync --extra xformers

OpenPoints is built through external/openpoints_shim during uv sync. If CUDA extension builds cannot find nvcc, set CUDA_HOME or OPENPOINTS_TORCH_CUDA_ARCH_LIST for the local machine.

1.3 Dataset and Cache Paths

ARIA-NBV defaults to repository-rooted paths:

Path Purpose
.data/aria_download_urls/ Project Aria download URL JSON files.
.data/ase_efm/<scene_id>/shards-*.tar ATEK EFM WebDataset shards.
.data/ase_meshes/scene_ply_<scene_id>.ply ASE ground-truth meshes.
.data/ase_meshes_processed/ Generated cropped or simplified meshes.
.data/offline_cache/vin_offline/ Default immutable VIN offline store.
.data/offline_cache/vin_offline_rerun_smoke_v7/ One-sample sidecar smoke store.
.logs/ckpts/ External model checkpoints.
.logs/checkpoints/ Lightning checkpoints produced by training.
.artifacts/rerun/ Saved Rerun .rrd recordings.
.artifacts/rollouts/ Rollout trace artifacts.

Relative immutable-store names such as vin_offline resolve under PathConfig().offline_cache_dir, whose default is .data/offline_cache. Use copied local TOML configs with absolute paths when a machine needs a different data mount.

This public setup page covers prerequisites, raw ASE access, and lightweight smoke commands. The active operator runbook for local and LRZ data generation recipes lives in aria_nbv/aria_nbv/data_handling/README.md; keep volatile campaign commands there rather than duplicating them in Quarto.

1.4 ASE Access and Weights

Request ASE access through the Project Aria ASE download page, then place the download manifests here:

.data/aria_download_urls/ase_mesh_download_urls.json
.data/aria_download_urls/AriaSyntheticEnvironment_ATEK_download_urls.json

List available GT-mesh scenes and download a small local subset:

cd aria_nbv
uv run nbv-downloader -m list -c efm -n 10
uv run nbv-downloader -m download -c efm --ns 1 --max-shards 1

External checkpoints belong in .logs/ckpts/:

.logs/ckpts/model_lite.pth
.logs/ckpts/dinov2_vitb14_reg4_pretrain.pth
.logs/ckpts/s3dis-train-pointnext-s-ngpus1-seed1742-20220525-162639-Gz4ViiL95b6KP9MMH2ytG3_ckpt_best.pth

1.5 Offline Store and VIN Diagnostics

Validate the default VIN offline store before training from it:

cd aria_nbv
uv run nbv-summary --config-path offline_only.toml

The summary command forces run_mode = "summarize_vin", disables W&B, uses one batch, and reads the configured offline store. The current default config points at vin_offline. Treat a manifest with "interrupted": true as a diagnostic store rather than a full training store.

Build immutable stores through the writer CLI:

cd aria_nbv
uv run nbv-build-offline --config-path ../.configs/build_vin_offline_81286.toml --dry-run
uv run nbv-build-offline --config-path ../.configs/build_vin_offline_81286.toml

1.6 One-Scene Smoke

Use the focused one-scene smoke checks before touching larger stores. The data-handling README owns the active local and LRZ command recipes; the sidecar smoke config writes one sample to vin_offline_rerun_smoke_v7:

cd aria_nbv
uv run nbv-build-offline --config-path ../.configs/smoke/build_vin_offline_rerun_smoke_v7.toml --dry-run
uv run nbv-build-offline --config-path ../.configs/smoke/build_vin_offline_rerun_smoke_v7.toml
uv run nbv-rerun-inspect --config-path ../.configs/smoke/rerun_offline_smoke_v7.toml --split val --index 0 --save ../.artifacts/rerun/offline_smoke_v7.rrd

The Rerun inspector is diagnostic-only. Its downsampling is display-only and must not be treated as label, ranking, or training behavior.

For local rollout-store plumbing, use the current V1 rollout builder smoke config:

cd aria_nbv
uv run nbv-build-rollouts --config-path ../.configs/smoke/build_rollouts_v1_smoke.toml --dry-run

1.7 Stored Dataset Inspector

Open the Streamlit app with uv run nbv-st, then choose VIN Offline Dataset or Stored Rollouts. The stored-rollout inspector is organized as five native, lazy Streamlit tabs: Trust & Topology, Scientific Evidence, Targets & Action Support, Failure Triage, and Inspect, Export & Rerun. Tab changes rerun the app, but only the open tab performs its expensive projections. Immutable-store inventories and normalized evidence projections are cached; use Refresh stores after creating or replacing a store.

Start with Trust & Topology. Its invariant matrix distinguishes actor-visible evidence, privileged oracle/evaluation artifacts, derived q_h training caches, and provenance. The topology resolves rollout lineage to VIN manifests by hash and reports embedded, resolved, lineage-only, inferred, missing, and ambiguous links without mutating any store. The same topology model is available from the VIN page.

Scientific Evidence keeps matched inference and descriptive temporal summaries separate. Paired policy comparisons are emitted only for exactly matched source, target protocol, horizon and acquisition budget, candidate/oracle configuration, and branch-schedule cohorts. If those keys differ, the inspector reports the mismatched keys instead of drawing an unpaired policy comparison.

The temporal explorer displays one metric and unit at a time as the median and interquartile range over rollout depth. Quartiles use deterministic linear interpolation over finite rows only. Every point reports finite_count / total_count, missingness, mean, minimum, and maximum; missing depths are neither zero-filled nor interpolated. Upstream experiment dimensions such as persisted policy, horizon, and budget configuration are descriptive unless the matched-cohort gate passes. Selected-action provenance groups (selected_position, selected_strategy, and selected_mixture) are post-selection strata, not causal candidate-sampling effects. Raw connected lines are restricted to one explicitly selected rollout in the drill-down.

Targets & Action Support includes a complete, filtered Candidate provenance and support flow. This Sankey follows candidate rows—not rollout branches—from persisted mixture through position family, orientation strategy, actor-action validity, and selected/unselected outcome. Unknown persisted labels stay visible, and a selected actor-invalid row terminates at selection_contract_violation. Oracle-label and q_train masks are not shown as false sequential stages; their overlapping populations remain in the mask matrix. Policy and depth filters define the Sankey root denominator, and its CSV contains every aggregated link. Restored complete candidate breakdowns and bounded geometry, angle, motion, and reward plots remain explicit load-on-demand views. Candidate geometry is root-relative in the ARIA right-handed Z-up world frame and metres; unrelated absolute scene origins are never aggregated.

Every primary plot exposes its population, units, denominator, comparability requirements, interpretation, failure modes, and source fields through its information popover.

Every normalized table can be copied through Streamlit’s dataframe controls. Explicit downloads preserve the complete filtered CSV or JSON row set and state the exported count. In Inspect, Export & Rerun, the query workbench supports rollout summaries, factual steps, and candidates. Candidate scope defaults to the selected step; the selected rollout and explicit full store are available alternatives. Full-store candidate querying materializes the heavy candidate audit and therefore requires an explicit population choice.

Enter the expression portion of a regular pandas query, for example cumulative_target_root_gain > 0.5, selected_target_root_gain < 0, or position == "lateral_target_bypass" and actor_action. Apply query and Clear query are the only evaluation transitions; typing alone does not recompute the result. This is trusted local operator functionality over a copied normalized dataframe using DataFrame.query(..., engine="python"), not a security sandbox. An invalid expression reports the pandas error while preserving the last valid result and current rollout/step selection.

The workbench reports exact input, matched, displayed, and exported counts. Preview truncation affects only the displayed dataframe; the queried CSV always contains the complete matched result in deterministic order. Choose a matched row and use Promote queried row to carry its exact rollout and step into the Inspect selectors and Rerun launch form. A candidate promotes its owning step. The existing selected-step candidate download remains bounded to that selected rollout and step. The canonical evidence-bundle download is built lazily on click and defaults to pilot; confirmatory requires explicit acknowledgement.

Rerun launches use one of four layer presets—Scientific context, Target protocol, Candidate debugging, or Minimal selected path—with separate included and initially-visible settings. The launch surface supports native live spawn, save then open in the native viewer, save then --serve-web, and save-only recording. Web serving binds to loopback by default; LAN exposure is explicit. The page retains the resolved TOML, .rrd, stdout/stderr, process health, and verified viewer URL. Selected depth and target evaluation overlays remain visibly marked as privileged evaluation evidence.

1.8 Entry Points

cd aria_nbv

# Streamlit diagnostics app
uv run nbv-st

# VIN one-batch diagnostic summary
uv run nbv-summary --config-path offline_only.toml

# VIN training from the immutable offline store
uv run nbv-train --config-path offline_only.toml

# Fit and save the ordinal binner
uv run nbv-fit-binner --config-path offline_only.toml

# Dump resolved config
uv run nbv-cli --run-mode dump-config --config-path offline_only.toml

Use the Streamlit VIN diagnostics page to inspect offline-store coverage, tensor shapes, candidate/RRI alignment, and model debug outputs before trusting new training runs.

1.9 Docs Sanity

Run these after setup-doc or smoke-doc edits:

make qmd-frontmatter-check
cd docs
quarto render contents/setup.qmd
quarto render reference/index.qmd
quarto check
Back to top