1 rendering.unproject

rendering.unproject

Utilities to back-project rendered depth maps into world-frame point clouds.

This module centralises depth unprojection for candidate renders to avoid frame confusion between the PyTorch3D renderer (which outputs metric z depth in the physical camera frame) and downstream visualisations or fusion steps.

1.1 All functions assume

• depth is metric depth along the camera +Z axis (same convention as pytorch3d.renderer.MeshRasterizer with in_ndc=False).
• pose_world_cam is a efm3d.aria.pose.PoseTW storing world ← camera extrinsics (LUF camera frame).
• camera is the matching efm3d.aria.camera.CameraTW carrying intrinsics (and, if batched, per-candidate extrinsics that align with the provided depths/poses).

The returned points live in the same VIO/world frame as the ASE mesh and semidense history. Conceptually a valid depth pixel \(d(u,v)\) maps to \(\mathbf{p}_w = T_{w \leftarrow c}\,\pi^{-1}(u,v,d)\); crop and RRI code then decide whether the point contributes to scene-level or target-level scoring.

1.2 Functions

Name	Description
backproject_depths_camera_tw_batch	Back-project stored camera-z depth through the canonical PyTorch3D adapter.
backproject_depths_p3d_batch	Back-project a batch of PyTorch3D depth maps to world-frame points.

1.2.1 backproject_depths_camera_tw_batch

rendering.unproject.backproject_depths_camera_tw_batch(
    depths,
    mask_valid,
    camera,
    pose_world_camera,
    *,
    stride=1,
)

Back-project stored camera-z depth through the canonical PyTorch3D adapter.

1.2.2 backproject_depths_p3d_batch

rendering.unproject.backproject_depths_p3d_batch(
    depths,
    mask_valid,
    cameras,
    *,
    stride=1,
)

Back-project a batch of PyTorch3D depth maps to world-frame points.

1.2.2.1 Parameters

Name	Type	Description	Default
depths	torch.Tensor	Tensor["B", "H", "W"] metric z-depth maps in metres.	required
mask_valid	torch.Tensor	Tensor["B", "H", "W"] boolean masks for usable pixels.	required
cameras	PerspectiveCameras	One pytorch3d.renderer.PerspectiveCameras entry per depth map, carrying world-from-camera extrinsics.	required
stride	int	Pixel subsampling stride.	1

1.2.2.2 Returns

Name	Type	Description
	torch.Tensor	Pair (padded, lengths) where padded is
	torch.Tensor	Tensor["B", "Pmax", 3] in world coordinates and lengths is
	tuple[torch.Tensor, torch.Tensor]	Tensor["B"] with the valid count per candidate.