A Unified Framework for Multimodal, Prompt-Induced Emotional Text-to-Speech Synthesis
MM-TTS, a unified framework that leverages emotional cues from multiple modalities to generate highly expressive and emotionally resonant speech.
Cooperative Sentiment Agents for Multimodal Representation Learning and Sentiment Analysis
A novel Multimodal Representation Learning (MRL) method called Cooperative Sentiment Agents (Co-SA) that facilitates adaptive interaction between modalities to learn the joint representation for multimodal sentiment analysis.
IISAN: Efficiently Adapting Multimodal Representation for Sequential Recommendation with Decoupled Parameter-Efficient Fine-Tuning
The core message of this paper is that the authors propose a novel Intra- and Inter-modal Side Adapted Network (IISAN) that follows a decoupled parameter-efficient fine-tuning (DPEFT) paradigm to efficiently adapt pre-trained large-scale multimodal foundation models for downstream sequential recommendation tasks. IISAN significantly reduces GPU memory usage and training time compared to full fine-tuning and existing embedded PEFT methods, while maintaining comparable recommendation performance.
Large-Scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation
Proposing a pipeline for contrastive language-audio pretraining to enhance audio representation by combining audio data with natural language descriptions.
© 2024 by Linnk AI