Temel Kavramlar
Pre-trained models with prompt tuning can enhance generalized features for Universal Cross-Domain Retrieval, as demonstrated by the ProS method.
Özet
ProS introduces a novel method for Universal Cross-Domain Retrieval (UCDR) by leveraging prompt tuning to mine generalized knowledge from CLIP. The method involves a two-stage process: Prompt Units Learning (PUL) and Context-aware Simulator Learning (CSL). ProS achieves state-of-the-art performance on benchmark datasets without excessive parameters.
- Introduction
- Universal Cross-Domain Retrieval (UCDR) aims for robust performance in generalized test scenarios.
- Existing methods heavily rely on close-set learning settings, limiting model generalization.
- UCDR addresses domain and semantic shift challenges for model empowerment.
- Related Work
- UCDR combines Domain Generalization (DG) and Zero-Shot Learning (ZSL) approaches.
- Vision-Language Pre-training models like CLIP have shown promising results in UCDR tasks.
- Prompt Tuning methods offer a flexible strategy for adapting pre-trained models to downstream tasks.
- Method
- ProS introduces Prompt Units Learning (PUL) and Context-aware Simulator Learning (CSL) stages.
- PUL captures domain and semantic knowledge, while CSL trains a Content-aware Prompt Simulator.
- Retrieval using ProS involves generating Content-aware Dynamic Prompts (CaDP) for CLIP image features.
- Experiment
- ProS outperforms existing UCDR methods on benchmark datasets.
- Ablation studies confirm the importance of Semantic Prompt Units, Domain Prompt Units, and mask operations.
- ViT-based Context-aware Prompt Simulator with 2 layers yields optimal performance.
- Conclusion
- ProS demonstrates the effectiveness of prompt tuning for UCDR, achieving strong performance in generalized scenarios.
İstatistikler
ProS achieves a 22.27% improvement in mAP@200 compared to ViT.
ProS uses considerably fewer learnable parameters compared to full fine-tuning.
Alıntılar
"Prompt tuning can enhance the model's ability to tackle the UCDR task."
"Our method achieves new state-of-the-art performance without excessive parameters."