Core Concepts
This paper introduces a novel end-to-end learning framework to optimize acquisition-time wavefront modulations, which significantly enhance the ability to recover scenes obscured by scattering media.
Abstract
The paper presents a novel end-to-end learning framework that jointly optimizes wavefront modulations and a computationally lightweight feedforward "proxy" reconstruction network. The learned modulations are shown to generalize effectively to unseen scattering scenarios and exhibit remarkable versatility. During deployment, the learned modulations can be decoupled from the proxy network to augment other more computationally expensive restoration algorithms. Through extensive experiments on both simulated and real data, the approach is demonstrated to significantly advance the state of the art in imaging through scattering media.
The key highlights and insights are:
- The paper introduces a novel end-to-end learning framework to optimize wavefront modulations for imaging through scattering media.
- The jointly optimized proxy reconstruction network allows for efficient learning of the modulations, which can then be decoupled and used to enhance other reconstruction algorithms.
- Experiments on simulated and real data show that the learned modulations significantly outperform randomly chosen modulations or no modulations, for both data-driven and iterative optimization-based reconstruction methods.
- The learned modulations exhibit remarkable generalization capability, performing well on out-of-distribution target scenes and dynamic scenes.
Stats
The paper reports the following key metrics:
On simulated data, the learned modulations achieve a PSNR of 30.391 dB and an SSIM of 0.9082, outperforming the baselines of no modulations (PSNR 25.476 dB, SSIM 0.7640) and random modulations (PSNR 26.439 dB, SSIM 0.7980).
On real static in-distribution adipose tissue data, the proxy network with learned modulations achieves a PSNR of 19.06 dB and an SSIM of 0.58, compared to 17.57 dB PSNR and 0.48 SSIM for random modulations.
On real static out-of-distribution data, the unsupervised iterative approach [8] with learned modulations achieves a PSNR of 12.73 dB and an SSIM of 0.32, compared to 10.64 dB PSNR and 0.23 SSIM for random modulations.
On real dynamic out-of-distribution data, the unsupervised iterative approach [8] with learned modulations achieves a PSNR of 12.89 dB and an SSIM of 0.33, compared to 8.90 dB PSNR and 0.23 SSIM for random modulations.
Quotes
"Our learned modulations significantly enhance the reconstruction capability of both our data-driven proxy network and a state-of-the-art iterative optimization-based method [8] for imaging through scattering."
"Crucially, this proxy feedforward network allows us to bypass the expensive backpropagation computation through an iterative reconstruction algorithm, which would be necessary, but hopelessly impractical, if we were to naively optimize the modulations: each iteration would require us to finish an entire iterative reconstruction algorithm."