Efficient Multi-Task Learning with Low-Rank Adaptation (MTLoRA)

MTLoRA, a novel framework for parameter-efficient training of multi-task learning models, effectively balances learning shared and task-specific features during fine-tuning by employing Task-Agnostic and Task-Specific low-rank adaptation modules.