登入
洞見 - LoRAMoE Framework
LoRAMoE: Addressing World Knowledge Forgetting in Large Language Models
Large-scale increases in instruction data can lead to world knowledge forgetting in LLMs, but LoRAMoE mitigates this issue while enhancing multitasking abilities.
LoRAMoE: Addressing World Knowledge Forgetting in Large Language Models
Large-scale instruction data can damage world knowledge in LLMs, LoRAMoE mitigates this issue while enhancing downstream task performance.
LoRAMoE: Alleviating World Knowledge Forgetting in Large Language Models via MoE-Style Plugin
LoRAMoE introduces a novel framework to address the conflict between improving LLM performance on downstream tasks and preventing world knowledge forgetting during SFT.