Active 3DGS and Targeted NBV

1 Active 3DGS and Targeted NBV

Primary sources. ActiveNeRF [1], FisherRF [2], Next Best Sense [3], semantic/dynamic 3DGS NBV [4], object-centric 3DGS NBV [5], and FOV-HPE [6].

Source status. The corpus has local TeX mirrors for several 3DGS/NBV papers, including FisherRF, Dynamic 3DGS, Next Best Sense, and Instance/Object-centric NBV. FOV-HPE is tracked through DOI/PDF evidence, not a local TeX mirror.

Related ARIA-NBV pages. RRI theory, RL planning, and target-aware thesis questions.

1.1 Core contribution

Active NeRF and 3DGS papers are useful because they separate view utility into uncertainty, information, semantics, object focus, dynamics, or downstream-task loss. They do not replace ARIA-NBV’s utility target. The thesis should keep RRI and target RRI as labels/evaluation, and use 3DGS-style signals as proposal or diagnostic channels.

method	verified paper signal	ARIA-NBV adoption	do not adopt
ActiveNeRF [1]	Sparse-view NeRF acquisition uses uncertainty to choose views under a limited budget.	Use uncertainty-like scores as candidate proposal features and rank-agreement diagnostics.	Do not replace mesh-supervised RRI with NeRF uncertainty unless calibrated against RRI.
FisherRF [2]	Fisher information estimates how much a candidate view should reduce radiance-field uncertainty.	Treat Fisher-style information gain as a proposal channel for target-local evidence and uncertainty.	Do not treat Fisher information as GT reconstruction-quality improvement.
Next Best Sense [3]	Robotic 3DGS active sensing combines semantic depth alignment with Fisher-style depth uncertainty for view/touch selection.	Borrow the separation between representation uncertainty, semantic relevance, and downstream diagnostic value.	Do not import touch sensing or 3DGS scene state as thesis-core dependencies.
Semantic/dynamic 3DGS NBV [4]	Active selection can score geometry, semantic Gaussian parameters, and dynamic/deformation parameters separately.	Report scene RRI, target RRI, validity, semantic relevance, and motion cost as separate channels.	Do not make dynamic 3DGS a dependency for static ASE target-conditioned reconstruction.
Object-centric 3DGS NBV [5]	Instance/object features can focus view utility on underexplored target regions.	Adopt explicit target object, object-conditioned utility, and separate target metrics.	Do not use 3DGS object vectors as the first target representation; start with EVL/predicted OBB support.
FOV-HPE [6]	DOI/PDF evidence describes dynamic 3DGS novel-view rendering and RL viewpoint refinement for monocular 3D human-pose error.	Treat as evidence that 3DGS can become a downstream-task simulator bridge.	Do not use MPJPE, human-pose reward, or dynamic-human scenes as thesis-core objectives.

1.2 Verified paper signals

The common transferable structure is utility-channel separation:

\[ U(q) = \lambda_{\mathrm{geom}} U_{\mathrm{geom}}(q) + \lambda_{\mathrm{target}} U_{\mathrm{target}}(q) + \lambda_{\mathrm{unc}} U_{\mathrm{unc}}(q) + \lambda_{\mathrm{task}} U_{\mathrm{task}}(q). \]

ARIA-NBV should not collapse these into one opaque score. The thesis-core version should log them separately:

ARIA-NBV channel	role
scene RRI	global reconstruction-quality label/evaluation
target RRI	target-conditioned label/evaluation
mask and invalid reason	feasibility constraint
uncertainty / Fisher / evidence count	proposal or diagnostic signal
motion/path cost	optional planning penalty after quality evidence is trusted

1.3 ARIA-NBV adoption

Proposal/diagnostic: uncertainty, Fisher information, object focus, semantic relevance, rank agreement, and candidate diversity.
Gated follow-up: radiance-field or 3DGS scene state only after the finite-candidate target-RRI path works.
Stretch/bridge: 3DGS-backed continuous-control or downstream-task simulators after an environment abstraction exists.

1.4 Do not adopt

Do not move the thesis objective from target RRI to 3DGS uncertainty.
Do not make 3DGS a required reconstruction backend for ASE oracle rollout generation.
Do not claim local TeX verification for FOV-HPE; the current corpus has DOI/PDF evidence only.

1.5 Open risks / caveats

3DGS uncertainty and Fisher metrics can disagree with mesh-supervised reconstruction quality.
Target-aware 3DGS methods often assume representation state and optimization loops that ARIA-NBV does not yet have.
These papers are most useful as target/proposal inspiration, not as replacements for the finite-candidate Q_H thesis path.

References

[1]

X. Pan, Z. Lai, S. Song, and G. Huang, “ActiveNeRF: Learning where to see with uncertainty estimation.” 2022. Available: https://arxiv.org/abs/2209.08546

[2]

W. Jiang, B. Lei, and K. Daniilidis, “FisherRF: Active view selection and uncertainty quantification for radiance fields using fisher information.” 2024. Available: https://arxiv.org/abs/2311.17874

[3]

M. Strong, B. Lei, A. Swann, W. Jiang, K. Daniilidis, and M. K. III, “Next best sense: Guiding vision and touch with FisherRF for 3D gaussian splatting.” 2024. Available: https://arxiv.org/abs/2410.04680

[4]

Y. Li, W. Jiang, and K. Daniilidis, “Next best view selections for semantic and dynamic 3D gaussian splatting.” 2025. Available: https://arxiv.org/abs/2512.22771

[5]

S. Jeong, E. Lee, J. Kim, and A. Kim, “Informative object-centric next best view for object-aware 3D gaussian splatting in cluttered scenes.” 2026. Available: https://arxiv.org/abs/2602.08266

[6]

W. G. Bae, S. Lee, and J. T. Lee, “Finding optimal viewpoints for monocular 3D human pose estimation in dynamic 3D gaussian splatting space,” in Proceedings of the IEEE international conference on advanced video and signal-based surveillance, 2025. doi: 10.1109/AVSS65446.2025.11149906.

--- title: "Active 3DGS and Targeted NBV" phase: thesis audience: public status: current owner: jan format: html --- ## Active 3DGS and Targeted NBV {#active-3dgs-targeted-nbv-literature} **Primary sources.** ActiveNeRF [@ActiveNeRF-pan2022], FisherRF [@FisherRF-jiang2024], Next Best Sense [@NextBestSense-strong2024], semantic/dynamic 3DGS NBV [@li2025bestviewselectionssemantic], object-centric 3DGS NBV [@ObjectCentricNBV-jeong2026], and FOV-HPE [@FOVHPE-bae2025]. **Source status.** The corpus has local TeX mirrors for several 3DGS/NBV papers, including FisherRF, Dynamic 3DGS, Next Best Sense, and Instance/Object-centric NBV. FOV-HPE is tracked through DOI/PDF evidence, not a local TeX mirror. **Related ARIA-NBV pages.** [RRI theory](../theory/rri_theory.qmd), [RL planning](rl_planning.qmd), and [target-aware thesis questions](../thesis/questions.qmd). ### Core contribution Active NeRF and {{< gls three-dimensional-gaussian-splatting >}} papers are useful because they separate view utility into uncertainty, information, semantics, object focus, dynamics, or downstream-task loss. They do not replace ARIA-NBV's utility target. The thesis should keep {{< gls relative-reconstruction-improvement >}} and {{< gls target-specific-rri >}} as labels/evaluation, and use 3DGS-style signals as proposal or diagnostic channels. | method | verified paper signal | ARIA-NBV adoption | do not adopt | |---|---|---|---| | ActiveNeRF [@ActiveNeRF-pan2022] | Sparse-view NeRF acquisition uses uncertainty to choose views under a limited budget. | Use uncertainty-like scores as candidate proposal features and rank-agreement diagnostics. | Do not replace mesh-supervised RRI with NeRF uncertainty unless calibrated against RRI. | | FisherRF [@FisherRF-jiang2024] | Fisher information estimates how much a candidate view should reduce radiance-field uncertainty. | Treat Fisher-style information gain as a proposal channel for target-local evidence and uncertainty. | Do not treat Fisher information as GT reconstruction-quality improvement. | | Next Best Sense [@NextBestSense-strong2024] | Robotic 3DGS active sensing combines semantic depth alignment with Fisher-style depth uncertainty for view/touch selection. | Borrow the separation between representation uncertainty, semantic relevance, and downstream diagnostic value. | Do not import touch sensing or 3DGS scene state as thesis-core dependencies. | | Semantic/dynamic 3DGS NBV [@li2025bestviewselectionssemantic] | Active selection can score geometry, semantic Gaussian parameters, and dynamic/deformation parameters separately. | Report scene RRI, target RRI, validity, semantic relevance, and motion cost as separate channels. | Do not make dynamic 3DGS a dependency for static ASE target-conditioned reconstruction. | | Object-centric 3DGS NBV [@ObjectCentricNBV-jeong2026] | Instance/object features can focus view utility on underexplored target regions. | Adopt explicit target object, object-conditioned utility, and separate target metrics. | Do not use 3DGS object vectors as the first target representation; start with EVL/predicted OBB support. | | FOV-HPE [@FOVHPE-bae2025] | DOI/PDF evidence describes dynamic 3DGS novel-view rendering and RL viewpoint refinement for monocular 3D human-pose error. | Treat as evidence that 3DGS can become a downstream-task simulator bridge. | Do not use MPJPE, human-pose reward, or dynamic-human scenes as thesis-core objectives. | ### Verified paper signals The common transferable structure is utility-channel separation: $$ U(q) = \lambda_{\mathrm{geom}} U_{\mathrm{geom}}(q) + \lambda_{\mathrm{target}} U_{\mathrm{target}}(q) + \lambda_{\mathrm{unc}} U_{\mathrm{unc}}(q) + \lambda_{\mathrm{task}} U_{\mathrm{task}}(q). $$ ARIA-NBV should not collapse these into one opaque score. The thesis-core version should log them separately: | ARIA-NBV channel | role | |---|---| | scene {{< gls relative-reconstruction-improvement >}} | global reconstruction-quality label/evaluation | | {{< gls target-specific-rri >}} | target-conditioned label/evaluation | | {{< gls validity-mask >}} and invalid reason | feasibility constraint | | uncertainty / Fisher / evidence count | proposal or diagnostic signal | | motion/path cost | optional planning penalty after quality evidence is trusted | ### ARIA-NBV adoption - **Proposal/diagnostic:** uncertainty, Fisher information, object focus, semantic relevance, rank agreement, and candidate diversity. - **Gated follow-up:** radiance-field or 3DGS scene state only after the finite-candidate target-RRI path works. - **Stretch/bridge:** 3DGS-backed continuous-control or downstream-task simulators after an environment abstraction exists. ### Do not adopt - Do not move the thesis objective from {{< gls target-specific-rri >}} to 3DGS uncertainty. - Do not make 3DGS a required reconstruction backend for ASE oracle rollout generation. - Do not claim local TeX verification for FOV-HPE; the current corpus has DOI/PDF evidence only. ### Open risks / caveats - 3DGS uncertainty and Fisher metrics can disagree with mesh-supervised reconstruction quality. - Target-aware 3DGS methods often assume representation state and optimization loops that ARIA-NBV does not yet have. - These papers are most useful as target/proposal inspiration, not as replacements for the finite-candidate {{< gls finite-horizon-q-function >}} thesis path.