Concepts de base
The core message of this paper is to improve the training efficiency of score-based diffusion models for image denoising by solving the log-density Fokker-Planck equation numerically to compute the score before training, and embedding the pre-computed score into the image to encourage faster training.
Résumé
The paper presents a method to efficiently denoise images by improving the training efficiency of score-based diffusion models. The key contributions are:
- Deriving a semi-explicit finite difference approximation scheme to solve the log-density Fokker-Planck (FP) equation numerically.
- Introducing "score embedding", which allows the numerical solution of the FP equation to be embedded into the feature space of the image through the transport ODE. This enables the network to learn from the pre-computed score, improving training efficiency.
The authors first formulate the log-density FP equation and discretize it using finite difference methods. They then solve the FP equation numerically using a semi-explicit scheme and a sparse linear system solver to efficiently compute the score. This pre-computed score is then embedded into the image using the transport ODE before training the score-matching network.
The authors demonstrate the effectiveness of their proposed method on the CIFAR10, ImageNet, and CelebA datasets. Compared to standard score-based diffusion models (DDPM and DDIM), their method achieves similar denoising quality but with significantly faster training times, up to 18.62 times speedup.
Stats
The paper presents the following key metrics:
Mean Squared Error (MSE)
Structural Similarity Index (SSIM)
Training time
Citations
"Our proposed method allows the network to learn from the score embedded in the feature space, thus improving training efficiency."
"We demonstrate through our numerical experiments the improved performance of our proposed method compared to standard score-based diffusion models. Our proposed method achieves a similar quality to the standard method meaningfully faster."