Improving Paraphrased Retrieval in Dual-Encoder Vision-Language Models by Adapting Pretrained Language Models

Adapting dual-encoder vision-language models with pretrained language models significantly improves the ranking similarity for paraphrased queries while maintaining zero-shot classification and retrieval performance.