1 rendering

rendering

Rendering utilities used by oracle RRI and rollout diagnostics.

This package facade exports candidate-depth DTOs and renderer configs/backends, plus depth-to-world-point-cloud conversion. Backend modules own rasterization and ray casting; pose generation supplies cameras, while RRI modules own target-aware scoring and validity interpretation.

Candidate renders are metric depth images generated from ASE/EFM meshes and candidate PoseTW cameras. PyTorch3D returns z-depth in the physical camera frame; unprojection converts valid pixels back into world-frame point clouds that can be joined with semidense history before point-mesh RRI scoring.

Rendering code is an oracle/evaluation dependency. Actor-visible datasets may store poses, masks, and selected diagnostics, but rendered GT-depth point clouds are supervision artifacts unless an experiment explicitly exposes them.

1.1 Package guide

aria_nbv.rendering owns calibrated candidate depth rendering and conversion of those depths into world-frame candidate point clouds. It adapts EFM camera calibration and world-from-camera poses to the rendering backend while preserving metric camera-z depth and candidate-row identity.

1.2 Reuse this package

Use the public package facade for the supported pipeline:

from aria_nbv.rendering import CandidateDepthRendererConfig, build_candidate_pointclouds

renderer = CandidateDepthRendererConfig().setup_target()
depths = renderer.render(sample=snippet, candidates=candidates)
candidate_points = build_candidate_pointclouds(snippet, depths)

CandidateDepthRendererConfig owns output sizing, candidate limits, device selection, and the PyTorch3D backend configuration. CandidateDepths retains the rendered poses, physical calibration, valid-pixel mask, and full-shell candidate_indices. build_candidate_pointclouds owns vectorized unprojection, semidense fusion, padding, and world-space bounds.

Use render_compact_indices when a caller has already selected explicit rows from the compact valid candidate table. Do not reconstruct camera transforms or unprojection in callers; the backend adapter and unproject.py own those frame conventions. Detailed shapes and failure contracts live in their public docstrings.

1.3 Consumers and boundaries

The public rendering path is composed by Oracle scoring and the scene-label pipeline. Candidate generation owns pose feasibility and full-shell provenance. Oracle scorers own target cropping, validity interpretation, and RRI. Rollout storage decides which selected diagnostics to retain.

Rendered mesh depth and backprojected point clouds are privileged Oracle or evaluation evidence. They are not actor-visible model inputs unless an experiment explicitly defines and verifies that data contract.

Pytorch3DDepthRenderer is the batched backend used by CandidateDepthRenderer. Efm3dDepthRenderer is a lower-level CPU ray-casting backend for direct diagnostic use; it is not interchangeable through the high-level renderer config. Prefer the high-level facade for candidate tables.

1.4 Proof

From aria_nbv/, run the tests for the changed contract:

uv run pytest tests/pose_generation/test_api_geometry_contracts.py \
  tests/oracle/test_scoring.py \
  tests/integration/test_oracle_rri_labeler_real_data.py

The real-data integration test requires the local ASE data and rendering environment. Focused unit tests remain the portable proof for DTO and adapter changes.

1.5 Optional candidate viewing crop

viewing_crop.crop_mesh_for_views retains whole mesh faces whose triangle bounds intersect the world AABB enclosing current candidate cameras and the expanded target OBB. This conservative selection preserves potential foreground occluders and boundary-crossing triangles; it is not a target-only crop or triangle clipping. It returns ViewingCropMesh with original face-row provenance and deterministic vertex remapping, without modifying the snippet.

Both renderer entry points accept optional keyword mesh_override. None renders the original full mesh; an override changes geometry only, preserving camera calibration, render limits and compact-to-shell joins. The target scorer owns opt-in configuration and halo derivation. Cropped candidate images are only scoring evidence: acquire the selected view from the original full mesh and canonically unproject it before constructing the next counterfactual state. The crop must be rebuilt from each step’s current candidate cameras.

Portable geometry and renderer checks:

uv run pytest tests/rendering/test_viewing_crop.py \
  tests/rendering/test_candidate_renderer_cpu_backend.py

1.6 Classes

Name	Description
CandidateDepthRenderer	High-level wrapper that renders depth for compact valid candidate poses.
CandidateDepthRendererConfig	Compose camera calibration, render sizing, and backend selection.
CandidateDepths	Typed result for candidate depth rendering.
CandidatePointClouds	Batched candidate point clouds plus fused semi-dense reconstruction.
Efm3dDepthRenderer	CPU depth renderer built on trimesh ray tracing.
Efm3dDepthRendererConfig	Configure correctness-oriented CPU ray casting for metric depth.
Pytorch3DDepthRenderer	Depth rendering backend based on PyTorch3D.
Pytorch3DDepthRendererConfig	Configuration for `Pytorch3DDepthRenderer`.

1.6.1 CandidateDepthRenderer

rendering.CandidateDepthRenderer(config)

High-level wrapper that renders depth for compact valid candidate poses.

1.6.1.1 Methods

Name	Description
rendered_camera_centers	Return world centres for exactly the configured rendered prefix.
render	Render the configured valid-candidate prefix.
render_compact_indices	Render depth maps for explicit compact-valid candidate rows.

1.6.1.1.1 rendered_camera_centers

rendering.CandidateDepthRenderer.rendered_camera_centers(candidates)

Return world centres for exactly the configured rendered prefix.

1.6.1.1.1.1 Parameters

Name	Type	Description	Default
candidates	CandidateSamplingResult	Compact valid-camera table with full-shell provenance.	required

1.6.1.1.1.2 Returns

Name	Type	Description
	torch.Tensor	World metres `Tensor["C_render 3", float]` on the renderer device,
	torch.Tensor	retaining the candidate pose dtype and compact row order. The same
	torch.Tensor	canonical camera selection and candidate cap are used by `render`;
	torch.Tensor	no mesh is accessed and no rasterization occurs. This query does not
	torch.Tensor	mutate candidates, and callers must not mutate returned positions.

1.6.1.1.1.3 Raises

Name	Type	Description
	ValueError	No candidate cameras are available.

1.6.1.1.2 render

rendering.CandidateDepthRenderer.render(
    sample,
    candidates,
    *,
    mesh_override=None,
)

Render the configured valid-candidate prefix.

mesh_override replaces geometry only, leaving calibrated cameras, near/far limits and shell indices unchanged. None uses the original snippet mesh. Cropped renders are scoring evidence; full-scene selected observations must be acquired separately before causal state updates.

1.6.1.1.3 render_compact_indices

rendering.CandidateDepthRenderer.render_compact_indices(
    sample,
    candidates,
    compact_indices,
    *,
    mesh_override=None,
)

Render depth maps for explicit compact-valid candidate rows.

1.6.1.1.3.1 Parameters

Name	Type	Description	Default
sample	`EfmSnippetView`	Snippet with an attached mesh.	required
candidates	CandidateSamplingResult	Full candidate sampling result.	required
compact_indices	torch.Tensor \| list[int] \| tuple[int, …]	Row indices into `candidates.views`. These are compact valid-candidate indices, not full-shell indices.	required
mesh_override	ViewingCropMesh \| None	Optional world-frame geometry replacement. None renders the original full mesh; calibration is unchanged.	`None`

1.6.1.1.3.2 Returns

Name	Type	Description
	CandidateDepths	Rendered depth batch aligned with `compact_indices` and carrying
	CandidateDepths	full-shell `candidate_indices` for joins.

1.6.2 CandidateDepthRendererConfig

rendering.CandidateDepthRendererConfig()

Compose camera calibration, render sizing, and backend selection.

Rendering is an oracle/evaluation stage: its mesh-derived depths must not be exposed as actor-visible state. max_candidates_final bounds work on the already compact valid-candidate table rather than changing validity.

1.6.2.1 Attributes

Name	Description
target_type	Return the high-level candidate renderer constructed by this config.
device	Geometry tensor device resolved under `PYTORCH3D_BACKEND` policy.
renderer	Nested renderer config; omitted devices inherit before provider admission.
max_candidates_final	Maximum number of already-pruned candidate views to render.
resolution_scale	Optional positive uniform scale applied to H,W for rendering.
output_width_px	Optional exact rendered width in pixels; overrides `resolution_scale` when paired with height.
output_height_px	Optional exact rendered height in pixels; overrides `resolution_scale` when paired with width.
verbosity	Verbosity level for logging.
is_debug	Enable detailed debug logging.

1.6.3 CandidateDepths

rendering.CandidateDepths(
    depths,
    depths_valid_mask,
    poses,
    reference_pose,
    candidate_indices,
    camera,
    p3d_cameras,
)

Typed result for candidate depth rendering.

candidate_indices maps rendered compact rows back to the full candidate shell. That link is required when writing target-RRI labels or selected heavy diagnostics into rollouts.zarr.

1.6.3.1 Attributes

Name	Description
depths	Metric camera z-depth `Tensor["C H W", float]` in metres.
depths_valid_mask	Face-hit and near/far mask `Tensor["C H W", bool]` for raw depths.
poses	Rendered world-from-camera poses with logical shape `(C, 12)`.
reference_pose	Physical world-from-reference rig pose with logical shape `(12,)`.
candidate_indices	Full-shell row indices `Tensor["C", int64]` for rendered candidates.
camera	Physical calibration and camera-from-reference extrinsics aligned over `C`.
p3d_cameras	PyTorch3D cameras with world-to-view transforms aligned over `C`.

1.6.3.2 Methods

Name	Description
to_serializable	Serialize this batch into a cache-friendly CPU payload.
from_serializable	Reconstruct one batch from a serialized payload.

1.6.3.2.1 to_serializable

rendering.CandidateDepths.to_serializable()

Serialize this batch into a cache-friendly CPU payload.

1.6.3.2.2 from_serializable

rendering.CandidateDepths.from_serializable(payload, *, device)

Reconstruct one batch from a serialized payload.

1.6.3.2.2.1 Parameters

Name	Type	Description	Default
payload	dict[str, object]	Serialized payload produced by `to_serializable`.	required
device	torch.device	Destination device for tensors and wrappers.	required

1.6.3.2.2.2 Returns

Name	Type	Description
	'CandidateDepths'	Reconstructed candidate-depth batch.

1.6.4 CandidatePointClouds

rendering.CandidatePointClouds(
    points,
    lengths,
    semidense_points,
    semidense_length,
    occupancy_bounds,
)

Batched candidate point clouds plus fused semi-dense reconstruction.

1.6.4.1 Attributes

Name	Description
points	Padded candidate points `Tensor["C P 3", float]` in world metres.
lengths	Valid point counts `Tensor["C", int64]` for each padded row.
semidense_points	Collapsed observed SLAM points `Tensor["K 3", float]` in world metres.
semidense_length	Observed point count `Tensor["1", int64]` for serialization symmetry.
occupancy_bounds	World bounds `Tensor["6", float]` ordered xmin/xmax/ymin/ymax/zmin/zmax.

1.6.4.2 Methods

Name	Description
to_serializable	Serialize this point-cloud batch into a cache-friendly CPU payload.
from_serializable	Reconstruct one point-cloud batch from a serialized payload.

1.6.4.2.1 to_serializable

rendering.CandidatePointClouds.to_serializable()

Serialize this point-cloud batch into a cache-friendly CPU payload.

1.6.4.2.2 from_serializable

rendering.CandidatePointClouds.from_serializable(payload, *, device)

Reconstruct one point-cloud batch from a serialized payload.

1.6.4.2.2.1 Parameters

Name	Type	Description	Default
payload	dict[str, object]	Serialized payload produced by `to_serializable`.	required
device	torch.device	Destination device for tensors.	required

1.6.4.2.2.2 Returns

Name	Type	Description
	'CandidatePointClouds'	Reconstructed candidate-pointcloud batch.

1.6.5 Efm3dDepthRenderer

rendering.Efm3dDepthRenderer(config)

CPU depth renderer built on trimesh ray tracing.

1.6.5.1 Attributes

Name	Description
device	Return the Torch device receiving rendered `Tensor["H W"]` maps.

1.6.5.2 Methods

Name	Description
render_depth	Render a depth map for a single pose.
render_batch	Render depth for a batch of poses (iterative CPU fallback).

1.6.5.2.1 render_depth

rendering.Efm3dDepthRenderer.render_depth(
    pose_world_cam,
    mesh,
    camera,
    *,
    frame_index=None,
    occupancy_extent=None,
)

Render a depth map for a single pose.

1.6.5.2.1.1 Parameters

Name	Type	Description	Default
pose_world_cam	`PoseTW`	`T_world_camera` pose to render from.	required
mesh	`Trimesh`	Trimesh instance in world coordinates.	required
camera	`CameraTW`	Camera intrinsics/extrinsics (`CameraTW`).	required
frame_index	int \| None	Optional frame index for the calib tensor; defaults to the last frame if omitted.	`None`
occupancy_extent	Tensor \| None	Optional `[6]` tensor `[xmin,xmax,ymin,ymax,zmin,zmax]` to expand bounds when synthesising proxy walls.	`None`

1.6.5.2.1.2 Returns

Name	Type	Description
	Tensor	Tensor[H, W] on `config.device` filled with `zfar` on misses.

1.6.5.2.2 render_batch

rendering.Efm3dDepthRenderer.render_batch(
    poses,
    mesh,
    camera,
    *,
    frame_index=None,
    occupancy_extent=None,
)

Render depth for a batch of poses (iterative CPU fallback).

1.6.6 Efm3dDepthRendererConfig

rendering.Efm3dDepthRendererConfig()

Configure correctness-oriented CPU ray casting for metric depth.

Proxy-wall and chunk controls compensate for incomplete ASE room meshes and cap host-memory use. The backend always traces in world geometry, then returns camera z-depth in metres on the configured Torch output device.

1.6.6.1 Attributes

Name	Description
target_type	Factory target for `BaseConfig.setup_target`.
device	Torch device for the returned depth tensor.
zfar	Depth value (metres) for rays that miss the mesh.
add_proxy_walls	If `True` seal missing walls with thin box proxies.
proxy_wall_area_threshold	Minimum fraction of expected wall area required to skip proxies.
proxy_eps	Epsilon (metres) for detecting faces near bounds.
chunk_rays	Number of rays processed per chunk to limit memory.
backend	Ray backend: `'auto'` (pyembree if available else native),
verbosity	Enable structured logging.
is_debug	Enable extra debug logging.

1.6.7 Pytorch3DDepthRenderer

rendering.Pytorch3DDepthRenderer(config)

Depth rendering backend based on PyTorch3D.

1.6.7.1 Methods

Name	Description
render	Rasterize metric z-depth for world-from-camera candidate poses.

1.6.7.1.1 render

rendering.Pytorch3DDepthRenderer.render(
    poses,
    mesh,
    camera,
    *,
    frame_index=None,
)

Rasterize metric z-depth for world-from-camera candidate poses.

1.6.7.1.1.1 Parameters

Name	Type	Description	Default
poses	`PoseTW`	Batched `PoseTW` world-from-camera transforms with logical shape `(C, 12)`.	required
mesh	tuple[torch.Tensor, torch.Tensor]	World-frame vertices `Tensor["V 3", float]` in metres and triangle indices `Tensor["F 3", int64]`.	required
camera	`CameraTW`	EFM camera calibration supplying physical-image intrinsics.	required
frame_index	int \| None	Optional calibration row. When omitted, `camera` must already be singleton or aligned one-to-one with `poses`.	`None`

1.6.7.1.1.2 Returns

Name	Type	Description
	Tensor	A tuple of raw z-buffer `Tensor["C H W", float]` in metres,
	Tensor	closest-face indices `Tensor["C H W", int64]` with negative
	`PerspectiveCameras`	values for raster misses, and aligned PyTorch3D cameras. The
	tuple[Tensor, Tensor, `PerspectiveCameras`]	caller combines face hits with near/far checks to form a mask.

1.6.8 Pytorch3DDepthRendererConfig

rendering.Pytorch3DDepthRendererConfig()

Configuration for Pytorch3DDepthRenderer.

1.6.8.1 Attributes

Name	Description
target_type	Return the batched PyTorch3D rasterizer constructed by this config.
device	Geometry tensor device resolved under `PYTORCH3D_BACKEND` policy.
zfar	Far clipping plane in metres used by downstream validity checks.
znear	Near clipping plane (metres); triangles closer than this are clipped.
cull_backfaces	If `True` drop triangles with normals pointing away from the camera.
blur_radius	Soft rasterizer blur radius; keep `0` for hard z-buffer.
bin_size	Rasterisation bin size in pixels.
max_faces_per_bin	Performance knob mirroring `RasterizationSettings.max_faces_per_bin`.
max_views_per_batch	Maximum cameras rasterized together, bounding replicated mesh memory.
dtype	Computation dtype; use float16/bfloat16 on CUDA for speed
is_debug	Enable debug logging on the renderer console.
verbosity	Enable `Console` logging.

1.7 Functions

Name	Description
build_candidate_pointclouds	Convert stacked depth maps into batched point clouds and fuse with SLAM.
candidate_depth_renderer_ui_bounds	Return domain-owned limits for interactive depth-renderer controls.

1.7.1 build_candidate_pointclouds

rendering.build_candidate_pointclouds(
    sample,
    batch,
    *,
    stride=1,
    prepared_sample=None,
)

Convert stacked depth maps into batched point clouds and fuse with SLAM.

1.7.2 candidate_depth_renderer_ui_bounds

rendering.candidate_depth_renderer_ui_bounds()

Return domain-owned limits for interactive depth-renderer controls.

Interactive limits bound UI allocation and render count while preserving the broader programmatic configuration contract for batch callers.

# rendering { #aria_nbv.rendering } `rendering` Rendering utilities used by oracle RRI and rollout diagnostics. This package facade exports candidate-depth DTOs and renderer configs/backends, plus depth-to-world-point-cloud conversion. Backend modules own rasterization and ray casting; pose generation supplies cameras, while RRI modules own target-aware scoring and validity interpretation. Candidate renders are metric depth images generated from ASE/EFM meshes and candidate `PoseTW` cameras. PyTorch3D returns z-depth in the physical camera frame; unprojection converts valid pixels back into world-frame point clouds that can be joined with semidense history before point-mesh RRI scoring. Rendering code is an oracle/evaluation dependency. Actor-visible datasets may store poses, masks, and selected diagnostics, but rendered GT-depth point clouds are supervision artifacts unless an experiment explicitly exposes them.  ## Package guide `aria_nbv.rendering` owns calibrated candidate depth rendering and conversion of those depths into world-frame candidate point clouds. It adapts EFM camera calibration and world-from-camera poses to the rendering backend while preserving metric camera-z depth and candidate-row identity. ## Reuse this package Use the public package facade for the supported pipeline: ```python from aria_nbv.rendering import CandidateDepthRendererConfig, build_candidate_pointclouds renderer = CandidateDepthRendererConfig().setup_target() depths = renderer.render(sample=snippet, candidates=candidates) candidate_points = build_candidate_pointclouds(snippet, depths) ``` `CandidateDepthRendererConfig` owns output sizing, candidate limits, device selection, and the PyTorch3D backend configuration. `CandidateDepths` retains the rendered poses, physical calibration, valid-pixel mask, and full-shell `candidate_indices`. `build_candidate_pointclouds` owns vectorized unprojection, semidense fusion, padding, and world-space bounds. Use `render_compact_indices` when a caller has already selected explicit rows from the compact valid candidate table. Do not reconstruct camera transforms or unprojection in callers; the backend adapter and `unproject.py` own those frame conventions. Detailed shapes and failure contracts live in their public docstrings. ## Consumers and boundaries The public rendering path is composed by [Oracle scoring](https://github.com/JanDuchscherer104/ARIA-NBV/blob/main/aria_nbv/aria_nbv/oracle/_scoring.py) and the [scene-label pipeline](https://github.com/JanDuchscherer104/ARIA-NBV/blob/main/aria_nbv/aria_nbv/oracle/pipelines/scene_labels.py). Candidate generation owns pose feasibility and full-shell provenance. Oracle scorers own target cropping, validity interpretation, and RRI. Rollout storage decides which selected diagnostics to retain. Rendered mesh depth and backprojected point clouds are privileged Oracle or evaluation evidence. They are not actor-visible model inputs unless an experiment explicitly defines and verifies that data contract. `Pytorch3DDepthRenderer` is the batched backend used by `CandidateDepthRenderer`. `Efm3dDepthRenderer` is a lower-level CPU ray-casting backend for direct diagnostic use; it is not interchangeable through the high-level renderer config. Prefer the high-level facade for candidate tables. ## Proof From `aria_nbv/`, run the tests for the changed contract: ```sh uv run pytest tests/pose_generation/test_api_geometry_contracts.py \ tests/oracle/test_scoring.py \ tests/integration/test_oracle_rri_labeler_real_data.py ``` The real-data integration test requires the local ASE data and rendering environment. Focused unit tests remain the portable proof for DTO and adapter changes. ## Optional candidate viewing crop `viewing_crop.crop_mesh_for_views` retains whole mesh faces whose triangle bounds intersect the world AABB enclosing current candidate cameras and the expanded target OBB. This conservative selection preserves potential foreground occluders and boundary-crossing triangles; it is not a target-only crop or triangle clipping. It returns `ViewingCropMesh` with original face-row provenance and deterministic vertex remapping, without modifying the snippet. Both renderer entry points accept optional keyword `mesh_override`. None renders the original full mesh; an override changes geometry only, preserving camera calibration, render limits and compact-to-shell joins. The target scorer owns opt-in configuration and halo derivation. Cropped candidate images are only scoring evidence: acquire the selected view from the original full mesh and canonically unproject it before constructing the next counterfactual state. The crop must be rebuilt from each step's current candidate cameras. Portable geometry and renderer checks: ```sh uv run pytest tests/rendering/test_viewing_crop.py \ tests/rendering/test_candidate_renderer_cpu_backend.py ```  ## Classes | Name | Description | | --- | --- | | [CandidateDepthRenderer](#aria_nbv.rendering.CandidateDepthRenderer) | High-level wrapper that renders depth for compact valid candidate poses. | | [CandidateDepthRendererConfig](#aria_nbv.rendering.CandidateDepthRendererConfig) | Compose camera calibration, render sizing, and backend selection. | | [CandidateDepths](#aria_nbv.rendering.CandidateDepths) | Typed result for candidate depth rendering. | | [CandidatePointClouds](#aria_nbv.rendering.CandidatePointClouds) | Batched candidate point clouds plus fused semi-dense reconstruction. | | [Efm3dDepthRenderer](#aria_nbv.rendering.Efm3dDepthRenderer) | CPU depth renderer built on trimesh ray tracing. | | [Efm3dDepthRendererConfig](#aria_nbv.rendering.Efm3dDepthRendererConfig) | Configure correctness-oriented CPU ray casting for metric depth. | | [Pytorch3DDepthRenderer](#aria_nbv.rendering.Pytorch3DDepthRenderer) | Depth rendering backend based on PyTorch3D. | | [Pytorch3DDepthRendererConfig](#aria_nbv.rendering.Pytorch3DDepthRendererConfig) | Configuration for `Pytorch3DDepthRenderer`. | ### CandidateDepthRenderer { #aria_nbv.rendering.CandidateDepthRenderer } ```python rendering.CandidateDepthRenderer(config) ``` High-level wrapper that renders depth for compact valid candidate poses. #### Methods | Name | Description | | --- | --- | | [rendered_camera_centers](#aria_nbv.rendering.CandidateDepthRenderer.rendered_camera_centers) | Return world centres for exactly the configured rendered prefix. | | [render](#aria_nbv.rendering.CandidateDepthRenderer.render) | Render the configured valid-candidate prefix. | | [render_compact_indices](#aria_nbv.rendering.CandidateDepthRenderer.render_compact_indices) | Render depth maps for explicit compact-valid candidate rows. | ##### rendered_camera_centers { #aria_nbv.rendering.CandidateDepthRenderer.rendered_camera_centers } ```python rendering.CandidateDepthRenderer.rendered_camera_centers(candidates) ``` Return world centres for exactly the configured rendered prefix. ###### Parameters {.doc-section .doc-section-parameters} | Name | Type | Description | Default | |------------|-------------------------------------------------------------------------------------|--------------------------------------------------------|------------| | candidates | [CandidateSamplingResult](`aria_nbv.pose_generation.types.CandidateSamplingResult`) | Compact valid-camera table with full-shell provenance. | _required_ | ###### Returns {.doc-section .doc-section-returns} | Name | Type | Description | |--------|-------------------------------------------|----------------------------------------------------------------------| | | [torch](`torch`).[Tensor](`torch.Tensor`) | World metres ``Tensor["C_render 3", float]`` on the renderer device, | | | [torch](`torch`).[Tensor](`torch.Tensor`) | retaining the candidate pose dtype and compact row order. The same | | | [torch](`torch`).[Tensor](`torch.Tensor`) | canonical camera selection and candidate cap are used by ``render``; | | | [torch](`torch`).[Tensor](`torch.Tensor`) | no mesh is accessed and no rasterization occurs. This query does not | | | [torch](`torch`).[Tensor](`torch.Tensor`) | mutate candidates, and callers must not mutate returned positions. | ###### Raises {.doc-section .doc-section-raises} | Name | Type | Description | |--------|----------------------------|-------------------------------------| | | [ValueError](`ValueError`) | No candidate cameras are available. | ##### render { #aria_nbv.rendering.CandidateDepthRenderer.render } ```python rendering.CandidateDepthRenderer.render( sample, candidates, *, mesh_override=None, ) ``` Render the configured valid-candidate prefix. ``mesh_override`` replaces geometry only, leaving calibrated cameras, near/far limits and shell indices unchanged. None uses the original snippet mesh. Cropped renders are scoring evidence; full-scene selected observations must be acquired separately before causal state updates. ##### render_compact_indices { #aria_nbv.rendering.CandidateDepthRenderer.render_compact_indices } ```python rendering.CandidateDepthRenderer.render_compact_indices( sample, candidates, compact_indices, *, mesh_override=None, ) ``` Render depth maps for explicit compact-valid candidate rows. ###### Parameters {.doc-section .doc-section-parameters} | Name | Type | Description | Default | |-----------------|----------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------|------------| | sample | [EfmSnippetView](`aria_nbv.data_handling.EfmSnippetView`) | Snippet with an attached mesh. | _required_ | | candidates | [CandidateSamplingResult](`aria_nbv.pose_generation.types.CandidateSamplingResult`) | Full candidate sampling result. | _required_ | | compact_indices | [torch](`torch`).[Tensor](`torch.Tensor`) \| [list](`list`)\[[int](`int`)\] \| [tuple](`tuple`)\[[int](`int`), ...\] | Row indices into ``candidates.views``. These are compact valid-candidate indices, not full-shell indices. | _required_ | | mesh_override | [ViewingCropMesh](`aria_nbv.rendering.viewing_crop.ViewingCropMesh`) \| None | Optional world-frame geometry replacement. None renders the original full mesh; calibration is unchanged. | `None` | ###### Returns {.doc-section .doc-section-returns} | Name | Type | Description | |--------|----------------------------------------------------------------------------------|--------------------------------------------------------------------| | | [CandidateDepths](`aria_nbv.rendering.candidate_depth_renderer.CandidateDepths`) | Rendered depth batch aligned with ``compact_indices`` and carrying | | | [CandidateDepths](`aria_nbv.rendering.candidate_depth_renderer.CandidateDepths`) | full-shell ``candidate_indices`` for joins. | ### CandidateDepthRendererConfig { #aria_nbv.rendering.CandidateDepthRendererConfig } ```python rendering.CandidateDepthRendererConfig() ``` Compose camera calibration, render sizing, and backend selection. Rendering is an oracle/evaluation stage: its mesh-derived depths must not be exposed as actor-visible state. ``max_candidates_final`` bounds work on the already compact valid-candidate table rather than changing validity. #### Attributes | Name | Description | | --- | --- | | [target_type](#aria_nbv.rendering.CandidateDepthRendererConfig.target_type) | Return the high-level candidate renderer constructed by this config. | | [device](#aria_nbv.rendering.CandidateDepthRendererConfig.device) | Geometry tensor device resolved under ``PYTORCH3D_BACKEND`` policy. | | [renderer](#aria_nbv.rendering.CandidateDepthRendererConfig.renderer) | Nested renderer config; omitted devices inherit before provider admission. | | [max_candidates_final](#aria_nbv.rendering.CandidateDepthRendererConfig.max_candidates_final) | Maximum number of already-pruned candidate views to render. | | [resolution_scale](#aria_nbv.rendering.CandidateDepthRendererConfig.resolution_scale) | Optional positive uniform scale applied to H,W for rendering. | | [output_width_px](#aria_nbv.rendering.CandidateDepthRendererConfig.output_width_px) | Optional exact rendered width in pixels; overrides ``resolution_scale`` when paired with height. | | [output_height_px](#aria_nbv.rendering.CandidateDepthRendererConfig.output_height_px) | Optional exact rendered height in pixels; overrides ``resolution_scale`` when paired with width. | | [verbosity](#aria_nbv.rendering.CandidateDepthRendererConfig.verbosity) | Verbosity level for logging. | | [is_debug](#aria_nbv.rendering.CandidateDepthRendererConfig.is_debug) | Enable detailed debug logging. | ### CandidateDepths { #aria_nbv.rendering.CandidateDepths } ```python rendering.CandidateDepths( depths, depths_valid_mask, poses, reference_pose, candidate_indices, camera, p3d_cameras, ) ``` Typed result for candidate depth rendering. `candidate_indices` maps rendered compact rows back to the full candidate shell. That link is required when writing target-RRI labels or selected heavy diagnostics into `rollouts.zarr`. #### Attributes | Name | Description | | --- | --- | | [depths](#aria_nbv.rendering.CandidateDepths.depths) | Metric camera z-depth ``Tensor["C H W", float]`` in metres. | | [depths_valid_mask](#aria_nbv.rendering.CandidateDepths.depths_valid_mask) | Face-hit and near/far mask ``Tensor["C H W", bool]`` for raw depths. | | [poses](#aria_nbv.rendering.CandidateDepths.poses) | Rendered world-from-camera poses with logical shape ``(C, 12)``. | | [reference_pose](#aria_nbv.rendering.CandidateDepths.reference_pose) | Physical world-from-reference rig pose with logical shape ``(12,)``. | | [candidate_indices](#aria_nbv.rendering.CandidateDepths.candidate_indices) | Full-shell row indices ``Tensor["C", int64]`` for rendered candidates. | | [camera](#aria_nbv.rendering.CandidateDepths.camera) | Physical calibration and camera-from-reference extrinsics aligned over ``C``. | | [p3d_cameras](#aria_nbv.rendering.CandidateDepths.p3d_cameras) | PyTorch3D cameras with world-to-view transforms aligned over ``C``. | #### Methods | Name | Description | | --- | --- | | [to_serializable](#aria_nbv.rendering.CandidateDepths.to_serializable) | Serialize this batch into a cache-friendly CPU payload. | | [from_serializable](#aria_nbv.rendering.CandidateDepths.from_serializable) | Reconstruct one batch from a serialized payload. | ##### to_serializable { #aria_nbv.rendering.CandidateDepths.to_serializable } ```python rendering.CandidateDepths.to_serializable() ``` Serialize this batch into a cache-friendly CPU payload. ##### from_serializable { #aria_nbv.rendering.CandidateDepths.from_serializable } ```python rendering.CandidateDepths.from_serializable(payload, *, device) ``` Reconstruct one batch from a serialized payload. ###### Parameters {.doc-section .doc-section-parameters} | Name | Type | Description | Default | |---------|----------------------------------------------------|---------------------------------------------------|------------| | payload | [dict](`dict`)\[[str](`str`), [object](`object`)\] | Serialized payload produced by `to_serializable`. | _required_ | | device | [torch](`torch`).[device](`torch.device`) | Destination device for tensors and wrappers. | _required_ | ###### Returns {.doc-section .doc-section-returns} | Name | Type | Description | |--------|---------------------|--------------------------------------| | | \'CandidateDepths\' | Reconstructed candidate-depth batch. | ### CandidatePointClouds { #aria_nbv.rendering.CandidatePointClouds } ```python rendering.CandidatePointClouds( points, lengths, semidense_points, semidense_length, occupancy_bounds, ) ``` Batched candidate point clouds plus fused semi-dense reconstruction. #### Attributes | Name | Description | | --- | --- | | [points](#aria_nbv.rendering.CandidatePointClouds.points) | Padded candidate points ``Tensor["C P 3", float]`` in world metres. | | [lengths](#aria_nbv.rendering.CandidatePointClouds.lengths) | Valid point counts ``Tensor["C", int64]`` for each padded row. | | [semidense_points](#aria_nbv.rendering.CandidatePointClouds.semidense_points) | Collapsed observed SLAM points ``Tensor["K 3", float]`` in world metres. | | [semidense_length](#aria_nbv.rendering.CandidatePointClouds.semidense_length) | Observed point count ``Tensor["1", int64]`` for serialization symmetry. | | [occupancy_bounds](#aria_nbv.rendering.CandidatePointClouds.occupancy_bounds) | World bounds ``Tensor["6", float]`` ordered xmin/xmax/ymin/ymax/zmin/zmax. | #### Methods | Name | Description | | --- | --- | | [to_serializable](#aria_nbv.rendering.CandidatePointClouds.to_serializable) | Serialize this point-cloud batch into a cache-friendly CPU payload. | | [from_serializable](#aria_nbv.rendering.CandidatePointClouds.from_serializable) | Reconstruct one point-cloud batch from a serialized payload. | ##### to_serializable { #aria_nbv.rendering.CandidatePointClouds.to_serializable } ```python rendering.CandidatePointClouds.to_serializable() ``` Serialize this point-cloud batch into a cache-friendly CPU payload. ##### from_serializable { #aria_nbv.rendering.CandidatePointClouds.from_serializable } ```python rendering.CandidatePointClouds.from_serializable(payload, *, device) ``` Reconstruct one point-cloud batch from a serialized payload. ###### Parameters {.doc-section .doc-section-parameters} | Name | Type | Description | Default | |---------|----------------------------------------------------|---------------------------------------------------|------------| | payload | [dict](`dict`)\[[str](`str`), [object](`object`)\] | Serialized payload produced by `to_serializable`. | _required_ | | device | [torch](`torch`).[device](`torch.device`) | Destination device for tensors. | _required_ | ###### Returns {.doc-section .doc-section-returns} | Name | Type | Description | |--------|--------------------------|-------------------------------------------| | | \'CandidatePointClouds\' | Reconstructed candidate-pointcloud batch. | ### Efm3dDepthRenderer { #aria_nbv.rendering.Efm3dDepthRenderer } ```python rendering.Efm3dDepthRenderer(config) ``` CPU depth renderer built on trimesh ray tracing. #### Attributes | Name | Description | | --- | --- | | [device](#aria_nbv.rendering.Efm3dDepthRenderer.device) | Return the Torch device receiving rendered ``Tensor["H W"]`` maps. | #### Methods | Name | Description | | --- | --- | | [render_depth](#aria_nbv.rendering.Efm3dDepthRenderer.render_depth) | Render a depth map for a single pose. | | [render_batch](#aria_nbv.rendering.Efm3dDepthRenderer.render_batch) | Render depth for a batch of poses (iterative CPU fallback). | ##### render_depth { #aria_nbv.rendering.Efm3dDepthRenderer.render_depth } ```python rendering.Efm3dDepthRenderer.render_depth( pose_world_cam, mesh, camera, *, frame_index=None, occupancy_extent=None, ) ``` Render a depth map for a single pose. ###### Parameters {.doc-section .doc-section-parameters} | Name | Type | Description | Default | |------------------|-----------------------------------|-------------------------------------------------------------------------------------------------------------|------------| | pose_world_cam | [PoseTW](`efm3d.aria.PoseTW`) | ``T_world_camera`` pose to render from. | _required_ | | mesh | [Trimesh](`trimesh.Trimesh`) | Trimesh instance in world coordinates. | _required_ | | camera | [CameraTW](`efm3d.aria.CameraTW`) | Camera intrinsics/extrinsics (``CameraTW``). | _required_ | | frame_index | [int](`int`) \| None | Optional frame index for the calib tensor; defaults to the last frame if omitted. | `None` | | occupancy_extent | [Tensor](`torch.Tensor`) \| None | Optional ``[6]`` tensor ``[xmin,xmax,ymin,ymax,zmin,zmax]`` to expand bounds when synthesising proxy walls. | `None` | ###### Returns {.doc-section .doc-section-returns} | Name | Type | Description | |--------|--------------------------|-------------------------------------------------------------------| | | [Tensor](`torch.Tensor`) | Tensor[H, W] on ``config.device`` filled with ``zfar`` on misses. | ##### render_batch { #aria_nbv.rendering.Efm3dDepthRenderer.render_batch } ```python rendering.Efm3dDepthRenderer.render_batch( poses, mesh, camera, *, frame_index=None, occupancy_extent=None, ) ``` Render depth for a batch of poses (iterative CPU fallback). ### Efm3dDepthRendererConfig { #aria_nbv.rendering.Efm3dDepthRendererConfig } ```python rendering.Efm3dDepthRendererConfig() ``` Configure correctness-oriented CPU ray casting for metric depth. Proxy-wall and chunk controls compensate for incomplete ASE room meshes and cap host-memory use. The backend always traces in world geometry, then returns camera z-depth in metres on the configured Torch output device. #### Attributes | Name | Description | | --- | --- | | [target_type](#aria_nbv.rendering.Efm3dDepthRendererConfig.target_type) | Factory target for `BaseConfig.setup_target`. | | [device](#aria_nbv.rendering.Efm3dDepthRendererConfig.device) | Torch device for the returned depth tensor. | | [zfar](#aria_nbv.rendering.Efm3dDepthRendererConfig.zfar) | Depth value (metres) for rays that miss the mesh. | | [add_proxy_walls](#aria_nbv.rendering.Efm3dDepthRendererConfig.add_proxy_walls) | If ``True`` seal missing walls with thin box proxies. | | [proxy_wall_area_threshold](#aria_nbv.rendering.Efm3dDepthRendererConfig.proxy_wall_area_threshold) | Minimum fraction of expected wall area required to skip proxies. | | [proxy_eps](#aria_nbv.rendering.Efm3dDepthRendererConfig.proxy_eps) | Epsilon (metres) for detecting faces near bounds. | | [chunk_rays](#aria_nbv.rendering.Efm3dDepthRendererConfig.chunk_rays) | Number of rays processed per chunk to limit memory. | | [backend](#aria_nbv.rendering.Efm3dDepthRendererConfig.backend) | Ray backend: ``'auto'`` (pyembree if available else native), | | [verbosity](#aria_nbv.rendering.Efm3dDepthRendererConfig.verbosity) | Enable structured logging. | | [is_debug](#aria_nbv.rendering.Efm3dDepthRendererConfig.is_debug) | Enable extra debug logging. | ### Pytorch3DDepthRenderer { #aria_nbv.rendering.Pytorch3DDepthRenderer } ```python rendering.Pytorch3DDepthRenderer(config) ``` Depth rendering backend based on PyTorch3D. #### Methods | Name | Description | | --- | --- | | [render](#aria_nbv.rendering.Pytorch3DDepthRenderer.render) | Rasterize metric z-depth for world-from-camera candidate poses. | ##### render { #aria_nbv.rendering.Pytorch3DDepthRenderer.render } ```python rendering.Pytorch3DDepthRenderer.render( poses, mesh, camera, *, frame_index=None, ) ``` Rasterize metric z-depth for world-from-camera candidate poses. ###### Parameters {.doc-section .doc-section-parameters} | Name | Type | Description | Default | |-------------|----------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------|------------| | poses | [PoseTW](`efm3d.aria.PoseTW`) | Batched ``PoseTW`` world-from-camera transforms with logical shape ``(C, 12)``. | _required_ | | mesh | [tuple](`tuple`)\[[torch](`torch`).[Tensor](`torch.Tensor`), [torch](`torch`).[Tensor](`torch.Tensor`)\] | World-frame vertices ``Tensor["V 3", float]`` in metres and triangle indices ``Tensor["F 3", int64]``. | _required_ | | camera | [CameraTW](`efm3d.aria.CameraTW`) | EFM camera calibration supplying physical-image intrinsics. | _required_ | | frame_index | [int](`int`) \| None | Optional calibration row. When omitted, ``camera`` must already be singleton or aligned one-to-one with ``poses``. | `None` | ###### Returns {.doc-section .doc-section-returns} | Name | Type | Description | |--------|-----------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------| | | [Tensor](`torch.Tensor`) | A tuple of raw z-buffer ``Tensor["C H W", float]`` in metres, | | | [Tensor](`torch.Tensor`) | closest-face indices ``Tensor["C H W", int64]`` with negative | | | [PerspectiveCameras](`pytorch3d.renderer.cameras.PerspectiveCameras`) | values for raster misses, and aligned PyTorch3D cameras. The | | | [tuple](`tuple`)\[[Tensor](`torch.Tensor`), [Tensor](`torch.Tensor`), [PerspectiveCameras](`pytorch3d.renderer.cameras.PerspectiveCameras`)\] | caller combines face hits with near/far checks to form a mask. | ### Pytorch3DDepthRendererConfig { #aria_nbv.rendering.Pytorch3DDepthRendererConfig } ```python rendering.Pytorch3DDepthRendererConfig() ``` Configuration for `Pytorch3DDepthRenderer`. #### Attributes | Name | Description | | --- | --- | | [target_type](#aria_nbv.rendering.Pytorch3DDepthRendererConfig.target_type) | Return the batched PyTorch3D rasterizer constructed by this config. | | [device](#aria_nbv.rendering.Pytorch3DDepthRendererConfig.device) | Geometry tensor device resolved under ``PYTORCH3D_BACKEND`` policy. | | [zfar](#aria_nbv.rendering.Pytorch3DDepthRendererConfig.zfar) | Far clipping plane in metres used by downstream validity checks. | | [znear](#aria_nbv.rendering.Pytorch3DDepthRendererConfig.znear) | Near clipping plane (metres); triangles closer than this are clipped. | | [cull_backfaces](#aria_nbv.rendering.Pytorch3DDepthRendererConfig.cull_backfaces) | If ``True`` drop triangles with normals pointing away from the camera. | | [blur_radius](#aria_nbv.rendering.Pytorch3DDepthRendererConfig.blur_radius) | Soft rasterizer blur radius; keep ``0`` for hard z-buffer. | | [bin_size](#aria_nbv.rendering.Pytorch3DDepthRendererConfig.bin_size) | Rasterisation bin size in pixels. | | [max_faces_per_bin](#aria_nbv.rendering.Pytorch3DDepthRendererConfig.max_faces_per_bin) | Performance knob mirroring ``RasterizationSettings.max_faces_per_bin``. | | [max_views_per_batch](#aria_nbv.rendering.Pytorch3DDepthRendererConfig.max_views_per_batch) | Maximum cameras rasterized together, bounding replicated mesh memory. | | [dtype](#aria_nbv.rendering.Pytorch3DDepthRendererConfig.dtype) | Computation dtype; use float16/bfloat16 on CUDA for speed | | [is_debug](#aria_nbv.rendering.Pytorch3DDepthRendererConfig.is_debug) | Enable debug logging on the renderer console. | | [verbosity](#aria_nbv.rendering.Pytorch3DDepthRendererConfig.verbosity) | Enable `Console` logging. | ## Functions | Name | Description | | --- | --- | | [build_candidate_pointclouds](#aria_nbv.rendering.build_candidate_pointclouds) | Convert stacked depth maps into batched point clouds and fuse with SLAM. | | [candidate_depth_renderer_ui_bounds](#aria_nbv.rendering.candidate_depth_renderer_ui_bounds) | Return domain-owned limits for interactive depth-renderer controls. | ### build_candidate_pointclouds { #aria_nbv.rendering.build_candidate_pointclouds } ```python rendering.build_candidate_pointclouds( sample, batch, *, stride=1, prepared_sample=None, ) ``` Convert stacked depth maps into batched point clouds and fuse with SLAM. ### candidate_depth_renderer_ui_bounds { #aria_nbv.rendering.candidate_depth_renderer_ui_bounds } ```python rendering.candidate_depth_renderer_ui_bounds() ``` Return domain-owned limits for interactive depth-renderer controls. Interactive limits bound UI allocation and render count while preserving the broader programmatic configuration contract for batch callers.