Analyzing and Reducing Catastrophic Forgetting in Parameter Efficient Tuning
The author explores the mode connectivity phenomenon in continual learning scenarios for large language models, aiming to strike a balance between plasticity and stability through innovative methods like I-LoRA.
