Centrala begrepp
The proposed learning paradigm leverages semantic discriminators and object-level discriminators to significantly improve the generation quality and realism of complex semantics and objects in structure-guided image completion tasks.
Sammanfattning
The paper presents a new learning paradigm for structure-guided image completion that aims to address the limitations of existing methods in hallucinating realistic object instances in complex natural scenes. The key contributions are:
- Semantic discriminators that leverage pretrained visual features to improve the realism of the generated visual concepts.
- Object-level discriminators that take aligned instances as inputs to enforce the realism of individual objects.
- State-of-the-art results on various tasks including segmentation-guided, edge-guided, and instance-guided image completion on the Places2 dataset.
- Flexibility of the trained model to support multiple editing use cases such as object insertion, replacement, removal, and standard inpainting.
- A novel automatic image completion pipeline that achieves state-of-the-art results on the standard inpainting task.
The paper first discusses related work on image inpainting and guided image inpainting. It then presents the proposed network architecture, including the generator, semantic discriminators, and object-level discriminators. The training objective and the fully automatic pipeline for standard inpainting are also described.
Extensive experiments are conducted on the Places2-person, Places2-object, and COCO-Stuff datasets, evaluating the model on instance-guided, segmentation-guided, and edge-guided inpainting tasks. Quantitative and qualitative results demonstrate the significant improvements in generation quality and realism compared to existing methods. The ablation study further highlights the importance of the proposed semantic and object-level discriminators.
Statistik
"Structure-guided image completion aims to inpaint a local region of an image according to an input guidance map from users."
"Existing methods often struggle to hallucinate realistic object instances in complex natural scenes."
"The proposed semantic discriminators leverage pretrained visual features to improve the realism of the generated visual concepts."
"The object-level discriminators take aligned instances as inputs to enforce the realism of individual objects."
"The trained model can support multiple editing use cases, such as object insertion, replacement, removal and standard inpainting."
"The proposed automatic image completion pipeline achieves state-of-the-art results on the standard inpainting task."
Citat
"Structure-guided image completion aims to inpaint a local region of an image according to an input guidance map from users."
"Existing methods often struggle to hallucinate realistic object instances in complex natural scenes."
"The proposed semantic discriminators leverage pretrained visual features to improve the realism of the generated visual concepts."
"The object-level discriminators take aligned instances as inputs to enforce the realism of individual objects."