Li, S., Liu, C., Zhang, T., Le, H., Süsstrunk, S., & Salzmann, M. (2024). Controlling the Fidelity and Diversity of Deep Generative Models via Pseudo Density. Transactions on Machine Learning Research.
This paper addresses the challenge of balancing fidelity and diversity in deep generative models by introducing a novel metric called "pseudo density" and proposing methods to control these aspects during both inference and fine-tuning.
The researchers propose a "pseudo density" metric that estimates the density of image data in a feature space extracted by a pre-trained image feature extractor. They utilize this metric to develop three techniques: 1) Per-sample perturbation of latent vectors to adjust realism and uniqueness of individual images. 2) Importance sampling during inference to control the proportion of high or low-density images. 3) Fine-tuning with importance sampling to guide the model towards learning an adjusted data distribution.
The study demonstrates the effectiveness of the proposed "pseudo density" metric and its associated techniques in controlling the fidelity and diversity of deep generative models. The authors highlight the importance of considering both fidelity and diversity in evaluating generative models, rather than solely relying on metrics like FID.
This research contributes significantly to the field of deep generative models by providing practical methods for controlling the quality and variety of generated images. This has implications for various applications, including image editing and generation, where fine-grained control over these aspects is crucial.
The paper acknowledges that further research could explore optimizing density-based sampling strategies and adapting the proposed control approach to conditional generation tasks like text-to-image synthesis.
Ke Bahasa Lain
dari konten sumber
arxiv.org
Pertanyaan yang Lebih Dalam