Audio-Driven Co-Speech Gesture Video Generation via Motion-Decoupled Diffusion Model
This work presents a novel motion-decoupled framework to directly generate audio-driven co-speech gesture videos, without relying on structural human priors. The key innovations include a nonlinear TPS transformation to extract latent motion features, a transformer-based diffusion model to capture the temporal correlation between gestures and speech, and a refinement network to enhance visual details.
Self-Supervised Learning of Latent Representation Deviations for Generating Realistic Co-speech Gesture Videos
A novel self-supervised approach to generate realistic co-speech gesture videos by learning deviations in the latent representation.
TANGO: A Novel Framework for Generating Realistic Co-Speech Gesture Videos Using Retrieval and Diffusion-Based Interpolation
TANGO is a novel framework that generates realistic and synchronized co-speech gesture videos by combining a hierarchical audio-motion embedding space for accurate gesture retrieval and a diffusion-based interpolation network for seamless transitions between retrieved video segments.
© 2024 by Linnk AI