
BRAVEn, an extension to the RAVEn method, learns strong visual and auditory speech representations entirely from raw audio-visual data, achieving state-of-the-art performance among self-supervised methods in various settings.


coremsg

braven-self-supervised-learning-of-robust-visual-and-auditory-speech-representations-from-raw-audio-visual-data


BRAVEn: Self-Supervised Learning of Robust Visual and Auditory Speech Representations from Raw Audio-Visual Data


title_rewrite


The proposed DCIM-AVSR model introduces an efficient asymmetric architecture that prioritizes the audio modality while treating the visual modality as supplementary, enabling more effective integration of multi-modal information through the Dual Conformer Interaction Module (DCIM).


efficient-audio-visual-speech-recognition-via-dual-conformer-interaction-module


Efficient Audio-Visual Speech Recognition via Dual Conformer Interaction Module
