Setup Instructions

1 Setup Instructions

This page mirrors the portable root setup guide in SETUP.md. Keep host-specific mounts and private machine notes out of public docs.

1.1 Clone and Environment

Clone the repository, initialize submodules, and build the package environment from aria_nbv/:

git clone https://github.com/JanDuchscherer104/ARIA-NBV.git
cd ARIA-NBV
git submodule update --init --recursive

cd aria_nbv
mamba env create -f environment.yml
mamba activate aria-nbv
uv sync --all-extras

environment.yml provides Python 3.11 plus a CUDA 12.1 build toolchain on Linux. If uv cannot resolve Python 3.11 automatically, pass a local interpreter:

cd aria_nbv
UV_PYTHON=/path/to/python3.11 uv sync --all-extras

Check the interpreter before diagnosing missing dependencies:

cd aria_nbv
uv run python --version
uv run python - <<'PY'
import torch
print("torch", torch.__version__)
print("cuda_available", torch.cuda.is_available())
print("cuda_version", torch.version.cuda)
PY

Torch CUDA availability is not enough for rollout/oracle rendering. PyTorch3D is a compiled extension and must pass its own CUDA rasterization smoke:

cd aria_nbv
uv run python - <<'PY'
import torch
from pytorch3d.renderer import FoVPerspectiveCameras, MeshRasterizer, RasterizationSettings
from pytorch3d.structures import Meshes

if not torch.cuda.is_available():
    raise SystemExit("Torch CUDA is unavailable.")

device = torch.device("cuda")
verts = torch.tensor([[-0.5, -0.5, 2.0], [0.5, -0.5, 2.0], [0.0, 0.5, 2.0]], device=device)
faces = torch.tensor([[0, 1, 2]], dtype=torch.int64, device=device)
mesh = Meshes(verts=[verts], faces=[faces])
rasterizer = MeshRasterizer(
    cameras=FoVPerspectiveCameras(device=device),
    raster_settings=RasterizationSettings(image_size=8, blur_radius=0.0, faces_per_pixel=1),
)
rasterizer(mesh)
print("pytorch3d_cuda_rasterization_ok")
PY

1.2 CUDA and CPU Expectations

CPU-only machines are suitable for docs work, config validation, lightweight tests, immutable-store reads, and rollout-builder dry-run validation. They are not the expected path for full ASE oracle generation, EVL backbone materialization, PyTorch3D depth rendering, or VIN training.

Linux GPU machines are the expected path for the thesis data loop. The package pins PyTorch CUDA 12.1 wheels on Linux, and several configs use device = "auto" so they can fall back to CPU for debugging. Treat CPU fallback as a diagnostic convenience rather than the performance target.

Use the aria-nbv mamba environment as the CUDA build-toolchain context and the repo .venv as the uv runtime. If PyTorch3D reports Not compiled with GPU support, rebuild it from the activated toolchain environment:

cd aria_nbv
mamba activate aria-nbv
export CUDA_HOME="$CONDA_PREFIX"
export FORCE_CUDA=1
export TORCH_CUDA_ARCH_LIST="8.6"
uv sync --all-extras --reinstall-package pytorch3d --no-build-isolation-package pytorch3d --no-cache

Optional xFormers can be installed on Linux when EFM3D attention utilities need it:

cd aria_nbv
uv sync --extra xformers

OpenPoints is built through external/openpoints_shim during uv sync. If CUDA extension builds cannot find nvcc, set CUDA_HOME or OPENPOINTS_TORCH_CUDA_ARCH_LIST for the local machine.

1.3 Dataset and Cache Paths

ARIA-NBV defaults to repository-rooted paths:

Path	Purpose
`.data/aria_download_urls/`	Project Aria download URL JSON files.
`.data/ase_efm/<scene_id>/shards-*.tar`	ATEK EFM WebDataset shards.
`.data/ase_meshes/scene_ply_<scene_id>.ply`	ASE ground-truth meshes.
`.data/ase_meshes_processed/`	Generated cropped or simplified meshes.
`.data/offline_cache/vin_offline/`	Default immutable VIN offline store.
`.data/offline_cache/vin_offline_rerun_smoke_v7/`	One-sample sidecar smoke store.
`.logs/ckpts/`	External model checkpoints.
`.logs/checkpoints/`	Lightning checkpoints produced by training.
`.artifacts/rerun/`	Saved Rerun `.rrd` recordings.
`.artifacts/rollouts/`	Rollout trace artifacts.

Relative immutable-store names such as vin_offline resolve under PathConfig().offline_cache_dir, whose default is .data/offline_cache. Use copied local TOML configs with absolute paths when a machine needs a different data mount.

This public setup page covers prerequisites, raw ASE access, and lightweight smoke commands. The active operator runbook for local and LRZ data generation recipes lives in aria_nbv/aria_nbv/data_handling/README.md; keep volatile campaign commands there rather than duplicating them in Quarto.

1.4 ASE Access and Weights

Request ASE access through the Project Aria ASE download page, then place the download manifests here:

.data/aria_download_urls/ase_mesh_download_urls.json
.data/aria_download_urls/AriaSyntheticEnvironment_ATEK_download_urls.json

List available GT-mesh scenes and download a small local subset:

cd aria_nbv
uv run nbv-downloader -m list -c efm -n 10
uv run nbv-downloader -m download -c efm --ns 1 --max-shards 1

External checkpoints belong in .logs/ckpts/:

.logs/ckpts/model_lite.pth
.logs/ckpts/dinov2_vitb14_reg4_pretrain.pth
.logs/ckpts/s3dis-train-pointnext-s-ngpus1-seed1742-20220525-162639-Gz4ViiL95b6KP9MMH2ytG3_ckpt_best.pth

1.5 Offline Store and VIN Diagnostics

Validate the default VIN offline store before training from it:

cd aria_nbv
uv run nbv-summary --config-path offline_only.toml

The summary command forces run_mode = "summarize_vin", disables W&B, uses one batch, and reads the configured offline store. The current default config points at vin_offline. Treat a manifest with "interrupted": true as a diagnostic store rather than a full training store.

Build immutable stores through the writer CLI:

cd aria_nbv
uv run nbv-build-offline --config-path ../.configs/build_vin_offline_81286.toml --dry-run
uv run nbv-build-offline --config-path ../.configs/build_vin_offline_81286.toml

1.6 One-Scene Smoke

Use the focused one-scene smoke checks before touching larger stores. The data-handling README owns the active local and LRZ command recipes; the sidecar smoke config writes one sample to vin_offline_rerun_smoke_v7:

cd aria_nbv
uv run nbv-build-offline --config-path ../.configs/build_vin_offline_rerun_smoke_v7.toml --dry-run
uv run nbv-build-offline --config-path ../.configs/build_vin_offline_rerun_smoke_v7.toml
uv run nbv-rerun-inspect --config-path ../.configs/rerun_offline_smoke_v7.toml --split val --index 0 --save ../.artifacts/rerun/offline_smoke_v7.rrd

The Rerun inspector is diagnostic-only. Its downsampling is display-only and must not be treated as label, ranking, or training behavior.

For local rollout-store plumbing, use the current V1 rollout builder smoke config:

cd aria_nbv
uv run nbv-build-rollouts --config-path ../.configs/build_rollouts_v1_smoke.toml --dry-run

1.7 Entry Points

cd aria_nbv

# Streamlit diagnostics app
uv run nbv-st

# VIN one-batch diagnostic summary
uv run nbv-summary --config-path offline_only.toml

# VIN training from the immutable offline store
uv run nbv-train --config-path offline_only.toml

# Fit and save the ordinal binner
uv run nbv-fit-binner --config-path offline_only.toml

# Dump resolved config
uv run nbv-cli --run-mode dump-config --config-path offline_only.toml

Use the Streamlit VIN diagnostics page to inspect offline-store coverage, tensor shapes, candidate/RRI alignment, and model debug outputs before trusting new training runs.

1.8 Docs Sanity

Run these after setup-doc or smoke-doc edits:

make qmd-frontmatter-check
cd docs
quarto render contents/setup.qmd
quarto render reference/index.qmd
quarto check

--- title: "Setup Instructions" phase: thesis audience: public status: current owner: jan format: html --- # Setup Instructions {#setup-instructions} This page mirrors the portable root setup guide in [`SETUP.md`](https://github.com/JanDuchscherer104/ARIA-NBV/blob/main/SETUP.md). Keep host-specific mounts and private machine notes out of public docs. ## Clone and Environment {#clone-and-environment} Clone the repository, initialize submodules, and build the package environment from `aria_nbv/`: ```sh git clone https://github.com/JanDuchscherer104/ARIA-NBV.git cd ARIA-NBV git submodule update --init --recursive cd aria_nbv mamba env create -f environment.yml mamba activate aria-nbv uv sync --all-extras ``` `environment.yml` provides Python 3.11 plus a CUDA 12.1 build toolchain on Linux. If `uv` cannot resolve Python 3.11 automatically, pass a local interpreter: ```sh cd aria_nbv UV_PYTHON=/path/to/python3.11 uv sync --all-extras ``` Check the interpreter before diagnosing missing dependencies: ```sh cd aria_nbv uv run python --version uv run python - <<'PY' import torch print("torch", torch.__version__) print("cuda_available", torch.cuda.is_available()) print("cuda_version", torch.version.cuda) PY ``` Torch CUDA availability is not enough for rollout/oracle rendering. PyTorch3D is a compiled extension and must pass its own CUDA rasterization smoke: ```sh cd aria_nbv uv run python - <<'PY' import torch from pytorch3d.renderer import FoVPerspectiveCameras, MeshRasterizer, RasterizationSettings from pytorch3d.structures import Meshes if not torch.cuda.is_available(): raise SystemExit("Torch CUDA is unavailable.") device = torch.device("cuda") verts = torch.tensor([[-0.5, -0.5, 2.0], [0.5, -0.5, 2.0], [0.0, 0.5, 2.0]], device=device) faces = torch.tensor([[0, 1, 2]], dtype=torch.int64, device=device) mesh = Meshes(verts=[verts], faces=[faces]) rasterizer = MeshRasterizer( cameras=FoVPerspectiveCameras(device=device), raster_settings=RasterizationSettings(image_size=8, blur_radius=0.0, faces_per_pixel=1), ) rasterizer(mesh) print("pytorch3d_cuda_rasterization_ok") PY ``` ## CUDA and CPU Expectations {#cuda-and-cpu-expectations} CPU-only machines are suitable for docs work, config validation, lightweight tests, immutable-store reads, and rollout-builder dry-run validation. They are not the expected path for full ASE oracle generation, EVL backbone materialization, PyTorch3D depth rendering, or VIN training. Linux GPU machines are the expected path for the thesis data loop. The package pins PyTorch CUDA 12.1 wheels on Linux, and several configs use `device = "auto"` so they can fall back to CPU for debugging. Treat CPU fallback as a diagnostic convenience rather than the performance target. Use the `aria-nbv` mamba environment as the CUDA build-toolchain context and the repo `.venv` as the `uv` runtime. If PyTorch3D reports `Not compiled with GPU support`, rebuild it from the activated toolchain environment: ```sh cd aria_nbv mamba activate aria-nbv export CUDA_HOME="$CONDA_PREFIX" export FORCE_CUDA=1 export TORCH_CUDA_ARCH_LIST="8.6" uv sync --all-extras --reinstall-package pytorch3d --no-build-isolation-package pytorch3d --no-cache ``` Optional xFormers can be installed on Linux when EFM3D attention utilities need it: ```sh cd aria_nbv uv sync --extra xformers ``` OpenPoints is built through `external/openpoints_shim` during `uv sync`. If CUDA extension builds cannot find `nvcc`, set `CUDA_HOME` or `OPENPOINTS_TORCH_CUDA_ARCH_LIST` for the local machine. ## Dataset and Cache Paths {#dataset-and-cache-paths} ARIA-NBV defaults to repository-rooted paths: | Path | Purpose | | --- | --- | | `.data/aria_download_urls/` | Project Aria download URL JSON files. | | `.data/ase_efm/<scene_id>/shards-*.tar` | ATEK EFM WebDataset shards. | | `.data/ase_meshes/scene_ply_<scene_id>.ply` | ASE ground-truth meshes. | | `.data/ase_meshes_processed/` | Generated cropped or simplified meshes. | | `.data/offline_cache/vin_offline/` | Default immutable VIN offline store. | | `.data/offline_cache/vin_offline_rerun_smoke_v7/` | One-sample sidecar smoke store. | | `.logs/ckpts/` | External model checkpoints. | | `.logs/checkpoints/` | Lightning checkpoints produced by training. | | `.artifacts/rerun/` | Saved Rerun `.rrd` recordings. | | `.artifacts/rollouts/` | Rollout trace artifacts. | Relative immutable-store names such as `vin_offline` resolve under `PathConfig().offline_cache_dir`, whose default is `.data/offline_cache`. Use copied local TOML configs with absolute paths when a machine needs a different data mount. This public setup page covers prerequisites, raw ASE access, and lightweight smoke commands. The active operator runbook for local and LRZ data generation recipes lives in `aria_nbv/aria_nbv/data_handling/README.md`; keep volatile campaign commands there rather than duplicating them in Quarto. ## ASE Access and Weights {#ase-access-and-weights} Request ASE access through the [Project Aria ASE download page](https://www.projectaria.com/datasets/ase/#download-dataset), then place the download manifests here: ```text .data/aria_download_urls/ase_mesh_download_urls.json .data/aria_download_urls/AriaSyntheticEnvironment_ATEK_download_urls.json ``` List available GT-mesh scenes and download a small local subset: ```sh cd aria_nbv uv run nbv-downloader -m list -c efm -n 10 uv run nbv-downloader -m download -c efm --ns 1 --max-shards 1 ``` External checkpoints belong in `.logs/ckpts/`: ```text .logs/ckpts/model_lite.pth .logs/ckpts/dinov2_vitb14_reg4_pretrain.pth .logs/ckpts/s3dis-train-pointnext-s-ngpus1-seed1742-20220525-162639-Gz4ViiL95b6KP9MMH2ytG3_ckpt_best.pth ``` ## Offline Store and VIN Diagnostics {#offline-store-and-vin-diagnostics} Validate the default VIN offline store before training from it: ```sh cd aria_nbv uv run nbv-summary --config-path offline_only.toml ``` The summary command forces `run_mode = "summarize_vin"`, disables W&B, uses one batch, and reads the configured offline store. The current default config points at `vin_offline`. Treat a manifest with `"interrupted": true` as a diagnostic store rather than a full training store. Build immutable stores through the writer CLI: ```sh cd aria_nbv uv run nbv-build-offline --config-path ../.configs/build_vin_offline_81286.toml --dry-run uv run nbv-build-offline --config-path ../.configs/build_vin_offline_81286.toml ``` ## One-Scene Smoke {#one-scene-smoke} Use the focused one-scene smoke checks before touching larger stores. The data-handling README owns the active local and LRZ command recipes; the sidecar smoke config writes one sample to `vin_offline_rerun_smoke_v7`: ```sh cd aria_nbv uv run nbv-build-offline --config-path ../.configs/build_vin_offline_rerun_smoke_v7.toml --dry-run uv run nbv-build-offline --config-path ../.configs/build_vin_offline_rerun_smoke_v7.toml uv run nbv-rerun-inspect --config-path ../.configs/rerun_offline_smoke_v7.toml --split val --index 0 --save ../.artifacts/rerun/offline_smoke_v7.rrd ``` The Rerun inspector is diagnostic-only. Its downsampling is display-only and must not be treated as label, ranking, or training behavior. For local rollout-store plumbing, use the current V1 rollout builder smoke config: ```sh cd aria_nbv uv run nbv-build-rollouts --config-path ../.configs/build_rollouts_v1_smoke.toml --dry-run ``` ## Entry Points {#entry-points} ```sh cd aria_nbv # Streamlit diagnostics app uv run nbv-st # VIN one-batch diagnostic summary uv run nbv-summary --config-path offline_only.toml # VIN training from the immutable offline store uv run nbv-train --config-path offline_only.toml # Fit and save the ordinal binner uv run nbv-fit-binner --config-path offline_only.toml # Dump resolved config uv run nbv-cli --run-mode dump-config --config-path offline_only.toml ``` Use the Streamlit VIN diagnostics page to inspect offline-store coverage, tensor shapes, candidate/RRI alignment, and model debug outputs before trusting new training runs. ## Docs Sanity {#docs-sanity} Run these after setup-doc or smoke-doc edits: ```sh make qmd-frontmatter-check cd docs quarto render contents/setup.qmd quarto render reference/index.qmd quarto check ```