核心概念
DerainNeRF simultaneously removes adhesive waterdrops from multi-view images and recovers the underlying clear 3D scene using a neural radiance fields (NeRF) representation.
要約
The paper proposes DerainNeRF, a method that addresses the challenge of 3D scene estimation from multi-view images degraded by adhesive waterdrops. The key components are:
- A pre-trained deep waterdrop detector is used to identify and localize waterdrops within the input images.
- DerainNeRF then leverages a NeRF-based network to estimate the clear 3D scene, excluding the waterdrop-covered pixels during training.
- The method can handle two scenarios: (a) waterdrops fixed to the scene while the camera moves, and (b) waterdrops fixed to the camera lens.
- Extensive experiments on both synthetic and real datasets demonstrate that DerainNeRF outperforms existing state-of-the-art image waterdrop removal methods in terms of rendering high-quality clear novel-view images.
- An ablation study shows that the mask enhancement based on average attention maps further improves the performance, especially when dealing with dense waterdrops.
統計
"When capturing images through the glass during rainy or snowy weather conditions, the resulting images often contain waterdrops adhered on the glass surface, and these waterdrops significantly degrade the image quality and performance of many computer vision algorithms."
"To tackle these limitations, we propose a method to reconstruct the clear 3D scene implicitly from multi-view images degraded by waterdrops."
引用
"Our method exploits an attention network to predict the location of waterdrops and then train a Neural Radiance Fields to recover the 3D scene implicitly."
"Extensive experimental results on both synthetic and real datasets show that our method is able to generate clear 3D scenes and outperforms existing state-of-the-art (SOTA) image adhesive waterdrop removal methods."