LoRAMoE: Addressing World Knowledge Forgetting in Large Language Models
LoRAMoE introduces a novel framework to address the conflict between improving downstream task performance and preventing world knowledge forgetting in large language models during supervised fine-tuning.
