Enhancing Parameter Efficiency of Low-Rank Adaptation (LoRA) through Weight Tying

Tied-LoRA, a novel paradigm that leverages weight tying and selective training, enhances the parameter efficiency of Low-Rank Adaptation (LoRA) while maintaining comparable performance across diverse tasks.