核心概念
GS-IR, a novel inverse rendering approach based on 3D Gaussian Splatting (3DGS), leverages forward mapping volume rendering to achieve photorealistic novel view synthesis and relighting results.
要約
The authors propose GS-IR, a novel inverse rendering framework that models a scene as a set of 3D Gaussians to achieve physically-based rendering and state-of-the-art decomposition results for both objects and scenes.
Key highlights:
- GS-IR addresses two main challenges when using 3DGS for inverse rendering: 1) 3DGS does not support producing plausible normals natively, and 2) forward mapping (e.g. rasterization and splatting) cannot trace the occlusion like backward mapping (e.g. ray tracing).
- To address the normal estimation issue, GS-IR proposes an efficient optimization scheme incorporating a depth-derivation-based regularization.
- To handle occlusion and model indirect lighting, GS-IR develops a baking-based method embedded in the framework.
- Experiments demonstrate the superiority of GS-IR over baseline methods in terms of both reconstruction quality and efficiency on various challenging scenes.
統計
"How can we deduce physical attributes (e.g. geometry, material, and lighting) of a 3D scene from multi-view images?"
"3D Gaussians are introduced as an unstructured scene representation to strike a balance between efficiency and quality."
"TensoIR [22] leverages the ray tracing of NeRF to directly model occlusion and indirect illumination."
引用
"Unlike previous works that use implicit neural representations and volume rendering (e.g. NeRF), which suffer from low expressive power and high computational complexity, we extend 3DGS, a top-performance representation for novel view synthesis, to estimate scene geometry, surface material, and environment illumination from multi-view images captured under unknown lighting conditions."
"To address these challenges, our GS-IR proposes an efficient optimization scheme incorporating a depth-derivation-based regularization for normal estimation and a baking-based occlusion to model indirect lighting."