Soft Momentum Contrastive Learning for Fine-grained Sentiment-aware Pre-training

The proposed SoftMCL introduces valence ratings as soft-label supervision for contrastive learning to fine-grained measure the sentiment similarities between samples, and performs contrastive learning on both word- and sentence-level to enhance the model's ability to learn affective information.