BiLoRA: A Bi-level Optimization Framework for Overfitting-Resilient Low-Rank Adaptation of Large Pre-trained Models

BiLoRA introduces a bi-level optimization framework to address overfitting in low-rank adaptation during fine-tuning, enhancing model generalization in natural language tasks.