Core Concepts
Half Fine-Tuning (HFT) allows large language models to acquire new abilities while retaining previously learned knowledge by selectively updating only half of the model parameters during fine-tuning.
Abstract
The paper introduces Half Fine-Tuning (HFT), a simple yet effective approach to mitigate catastrophic forgetting in large language models (LLMs) during fine-tuning.
The key insights are:
- Resetting half of the fine-tuned parameters to the pre-trained state can help restore some of the original knowledge while maintaining new learning abilities.
- HFT involves freezing half of the model parameters and only updating the other half during fine-tuning, without changing the model architecture.
- HFT can be seamlessly integrated into existing fine-tuning frameworks, including supervised fine-tuning, direct preference optimization, and continual learning.
Extensive experiments demonstrate the effectiveness of HFT:
- HFT significantly alleviates catastrophic forgetting compared to full fine-tuning (FFT), while achieving comparable or even better performance on downstream tasks.
- HFT is robust to the selection of trainable parameters, with around 50% of parameters being updated yielding the best results.
- HFT also improves training efficiency, reducing training time by approximately 30% compared to FFT.
The paper provides a theoretical interpretation of HFT from an optimization perspective, showing that it can be viewed as a form of regularization. The parameter variation analysis further reveals that HFT leads to more gradual changes in the model parameters compared to FFT.
Overall, HFT offers a simple yet powerful solution to preserve the knowledge of pre-trained LLMs while enabling effective fine-tuning for various tasks, making it a promising alternative to the standard fine-tuning approach.
Stats
Updating half of the parameters in LLAMA 2-CHAT-7B model can roughly restore a significant amount of forgotten basic knowledge while maintaining high-level general abilities performance.
Compared to full fine-tuning, HFT achieves an average performance improvement of 1.9% on LLAMA 2-7B and 2.9% on LLAMA 2-13B for general abilities benchmarks.
HFT consistently outperforms full fine-tuning and direct preference optimization in preserving basic knowledge, with improvements of 3.4% and 2.9% on LLAMA 2-7B and LLAMA 2-13B respectively.
HFT can shorten the training time by approximately 30% compared to full fine-tuning.
Quotes
"By regularly resetting partial parameters, LLMs can restore some of the original knowledge."
"Without changing the model architecture, HFT could be seamlessly integrated into existing fine-tuning frameworks."
"Extensive experiments and analysis demonstrate the effectiveness and efficiency of HFT."