The paper presents a novel Residual-Conditioned Optimal Transport (RCOT) approach for image restoration tasks, such as denoising, deraining, dehazing, and super-resolution. The key idea is to model the image restoration as an optimal transport (OT) problem and incorporate the degradation-specific knowledge from the transport residual into both the transport cost and the transport map.
Specifically, the authors first formulate a Fourier residual-guided OT (FROT) objective, which exploits the frequency statistics of the degradation domain gap (represented by the residual) to guide the transport cost. Then, they propose a two-pass RCOT map, in which the transport residual computed by the base model in the first pass is encoded as a degradation-specific embedding to condition the second-pass restoration. This conditioning mechanism enables the transport map to dynamically adjust its behavior for different restoration tasks and preserve the structural details of the restored images.
The authors evaluate the RCOT approach on various image restoration benchmarks, including denoising, deraining, dehazing, and super-resolution. The results demonstrate that RCOT outperforms state-of-the-art methods in terms of both distortion measures (PSNR, SSIM) and perceptual quality (LPIPS, FID), particularly in preserving the structural content of the restored images.
To Another Language
from source content
arxiv.org
Principais Insights Extraídos De
by Xiaole Tang,... às arxiv.org 05-07-2024
https://arxiv.org/pdf/2405.02843.pdfPerguntas Mais Profundas