ARIA-NBV: Target-Aware Next-Best-View Planning

Target-conditioned finite-candidate NBV with ASE, EFM3D, and Relative Reconstruction Improvement

Author

Jan Duchscherer, Munich University of Applied Sciences

Published

October 22, 2026

Explore the project: Research reports and issue map · Literature reviews · Thesis PDF · Roadmap

1 Abstract

Next-Best-View (NBV) planning addresses the fundamental challenge of autonomous viewpoint selection in active 3D reconstruction, aiming to maximize the acquisition quality while minimizing the acquisition cost (i.e. number of views, traversed distance, capture time).

Classical NBV methods rely on hand-crafted criteria, limited action spaces, or per-scene optimized representations. While learning-based NBV methods like GenNBV [1] have improved generalization by leveraging reinforcement learning, they still optimize for geometric coverage as a proxy for reconstruction quality. Since coverage maximization does not necessarily correlate with improved reconstruction quality, these methods can struggle in complex scenes with occlusions and fine details. Directly optimizing reconstruction quality, as pioneered by VIN-NBV [2], has shown significant improvements by predicting Relative Reconstruction Improvement (RRI) to quantify the fitness of candidate viewpoints. However, even VIN-NBV’s generalization capabilities are limited to simpler object-centric NBV scenarios because it does not leverage pre-trained foundation models with rich 3D spatial understanding.

ARIA-NBV studies whether target-conditioned, finite-candidate selection can improve endpoint reconstruction quality in complex indoor scenes. It uses ASE and the EVL (Egocentric Voxel Lifting) 3D EFM [3] as its egocentric substrate while keeping actor-visible observations separate from ground-truth geometry, target crops, and counterfactual renders used for supervision or evaluation. Current infrastructure covers oracle labels, immutable one-step stores, multi-step rollout persistence, inspection, and scorer-independent finite-horizon training contracts. Actor-visible target matching, a target-conditioned one-step scorer, and production \(Q_H\) evaluation remain active thesis gates.

2 Project Vision and Goals

2.1 Available infrastructure

  • Develop an oracle RRI computation pipeline using ASE visibility data, semi-dense point clouds, GT meshes, and the generated rri_metrics API contracts.
  • Generate and render finite candidate tables with hard validity masks and explicit invalid-reason codes.
  • Persist immutable VIN root evidence and target-conditioned rollout Zarr stores with source lineage, validation, and Streamlit/Rerun inspection surfaces.
  • Provide reader, dataset, collation, and scorer-independent fitted-\(Q\) training contracts for finite-horizon experiments.

2.2 Active evidence gates

  • Implement and validate observed or predicted target matching without treating privileged target geometry as an actor input.
  • Establish an actor-visible, target-conditioned one-step scorer as the myopic control.
  • Measure bounded oracle-lookahead headroom on trusted scene-disjoint support; evaluate a production \(Q_H\) scorer only when that gate is positive.
  • Treat online discrete learning, continuous or simulator-backed control, and real-device deployment as later work.

3 Documentation Navigation

The current thesis direction is owned by docs/typst/thesis/main.typ and its active sections; development-only status and gates are owned by the roadmap and M1 report. Executable behavior is owned by aria_nbv/aria_nbv/, aria_nbv/tests/, and active configuration; repository guidance and routing are owned by AGENTS.md and the aria-nbv-context owner hierarchy. The seminar paper and dated history remain evidence from earlier work and do not override these current owners.

3.1 Thesis and Seminar Paper

  • Thesis — DEVELOPMENT/DRAFT PDF, Typst Source: Tracked default development-mode artifact; roadmap, M1, and promotion-queue sections are intentionally present. A submission projection is separate and requires explicit submission mode plus the existing evidence gate.
  • Project Paper, Typst Source: Historical implemented evidence from the earlier project phase: Single-step RRI based scene scorer.

3.2 Project Slides

3.2.1 Project Presentations

3.3 Setup & Installation

3.4 Thesis State

3.5 Theory & Background

3.6 Dataset & Resources

3.7 Literature Reviews

  • Literature Review: Entry point and local LaTeX corpus
  • VIN-NBV: Direct quality optimization with RRI
  • GenNBV: Continuous action spaces and RL approaches
  • EFM3D & EVL: Egocentric foundation models and voxel lifting
  • SceneScript: Structured scene language and entity representation

3.8 API Reference

  • API Reference: Generated package topology for stable ARIA-NBV modules, including data handling, rendering, RRI metrics, training, and VIN scoring.
  • Setup Instructions: Environment, cache, and smoke commands for local validation.
  • Architecture Diagrams: Generated context diagrams and package-level views.
Back to top

References

[1]
X. Chen, Q. Li, T. Wang, T. Xue, and J. Pang, “GenNBV: Generalizable next-best-view policy for active 3D reconstruction,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2024, pp. 16436–16445. Available: https://openaccess.thecvf.com/content/CVPR2024/html/Chen_GenNBV_Generalizable_Next-Best-View_Policy_for_Active_3D_Reconstruction_CVPR_2024_paper.html
[2]
N. Frahm et al., “VIN-NBV: A view introspection network for next-best-view selection.” 2025. Available: https://arxiv.org/abs/2505.06219
[3]
J. Straub, D. DeTone, T. Shen, N. Yang, C. Sweeney, and R. Newcombe, “EFM3D: A benchmark for measuring progress towards 3D egocentric foundation models.” 2024. Available: https://arxiv.org/abs/2406.10224