Core Concepts
LMFlow is an extensible and lightweight toolkit that simplifies the domain- and task-aware finetuning of general foundation models, enabling efficient training and inference of specialized language models with limited computing resources.
Abstract
The paper introduces LMFlow, an extensible and lightweight toolkit for efficient finetuning and inference of large foundation models. The key highlights are:
LMFlow aims to simplify the domain- and task-aware finetuning of general foundation models, such as GPT-J, Bloom, and LLaMA, to support specialized training with limited computing resources.
It supports various finetuning techniques, including continuous pretraining, instruction tuning, parameter-efficient finetuning, alignment tuning, inference acceleration, long context generalization, model customization, and even multimodal finetuning.
LMFlow integrates efficient finetuning methods like LoRA, FlashAttention, and Gradient Checkpointing, as well as inference acceleration techniques like Speculative Decoding and LLaMA Inference on CPU.
The paper presents a novel alignment method called RAFT (Reward rAnked FineTuning) that utilizes a reward model to rank the output of the generative model, allowing for efficient and stable supervised finetuning-like training.
Experimental results demonstrate the effectiveness of LMFlow in training specialized language models, such as medical LLaMA, which can achieve comparable or even better performance than ChatGPT with significantly less training time and computing resources.
LMFlow is designed to be an extensible and easy-to-use toolkit, with carefully designed and extensible APIs, making it accessible for developers and researchers to train their own scientific or personalized language models.
Stats
It only takes one Nvidia 3090 GPU and five hours to train a medical LLaMA comparable to ChatGPT, based on a 7-billion-parameter LLaMA model.
The authors have released the model weights for medical LLaMA with 7-billion, 13-billion, 33-billion, and 65-billion parameters for academic research.
Quotes
"LMFlow offers a complete fine-tuning workflow for a foundation model to support specialized training with limited computing resources."
"LMFlow stands out as a comprehensive, full-cycle foundation model adaptation toolkit."