Comprehensive Insights into Parameter-Efficient Transfer Learning (PETL) for Visual Recognition

Parameter-efficient transfer learning (PETL) approaches can achieve similar accuracy to full fine-tuning while using much fewer learnable parameters. PETL approaches make different mistakes and high-confidence predictions, suggesting complementary information that can be leveraged through ensemble methods. PETL is also effective in many-shot regimes and better preserves the robustness of pre-trained models to distribution shifts.