Grunnleggende konsepter
PEGASUS, a method for constructing a personalized generative 3D face avatar from monocular video sources, enables disentangled controls to selectively alter facial attributes while preserving the identity.
Sammendrag
The paper presents PEGASUS, a method for building a personalized generative 3D face avatar from monocular video sources. The key highlights are:
- PEGASUS enables disentangled controls to selectively alter facial attributes (e.g., hair, nose, accessories) while preserving the identity of the target person.
- The approach consists of two stages: (1) synthetic database generation by borrowing facial attributes from diverse identities, and (2) constructing a personalized generative 3D avatar model.
- The synthetic database is created by swapping facial parts (e.g., hair, nose) of the target person with those from other monocular videos, allowing the model to learn continuous variations of facial attributes.
- The personalized generative 3D avatar model is trained on the synthetic database, enabling selective modification of facial attributes through disentangled latent codes.
- An alternative "zero-shot" approach is introduced to efficiently achieve the same goal of a personalized generative avatar by fusing the controlled attribute parts from a pre-trained model and the remaining parts of a new target identity.
- Extensive experiments demonstrate the effectiveness of the proposed methods in preserving identity while achieving high realism and controllability of facial attributes.
Statistikk
"Building a personalized 3D avatar for representing an individual in virtual spaces can bring significant advancements in the field of AR/VR and applications within the metaverse."
"Recent technologies make it possible to build high-quality 3D face avatars for general users from monocular video inputs only."
"Our personalized generative 3D avatar is constructed from the monocular video of the target individual."
Sitater
"In contrast to the previous work [59, 60], our 3D avatar enables compositional controls of facial attributes, where users can make alterations for desired facial attributes such as hair, nose, or accessories, as shown in Fig. 1, while preserving the identity of the target person."
"Built with diverse part-swapped videos, our generative 3D avatar, PEGASUS, can preserve high-quality details for the target individuals, while equipped with the generative power to selectively alter each facial part."