Kun: A Self-Training Approach for Generating High-Quality Chinese Instruction-Tuning Datasets for Large Language Models Using Instruction Back-Translation and Answer Polishment
Kun is a novel self-training method that leverages instruction back-translation and answer polishment to automatically generate large-scale, high-quality Chinese instruction-tuning datasets for LLMs, reducing the reliance on manual annotation.
IDEA-MCTS: A Framework for Optimizing Instruction Synthesis Using Monte Carlo Tree Search and Evaluation Models
This paper introduces IDEA-MCTS, a novel framework that leverages Monte Carlo Tree Search (MCTS) and evaluation models to automatically generate high-quality, diverse, and complex instruction data for improving the instruction-following abilities of large language models (LLMs).
인스트럭션 다양성이 일반화에 미치는 결정적 영향: ONLY-IF 연구
대규모 언어 모델의 효과적인 인스트럭션 튜닝을 위해서는 데이터셋의 양보다는 다양한 도메인을 포괄하는 인스트럭션의 다양성이 중요하다.
CommonIT: A New Instruction Tuning Method for Large Language Models Based on Data Partitioning
Commonality-aware Instruction Tuning (CommonIT) improves the instruction-following abilities of large language models by grouping similar training data into batches, enhancing the model's ability to learn and generalize across various tasks.
Don't Half-listen: Key-part Information in Continual Instruction Tuning
Proposing a novel continual instruction tuning method based on Key-part Information Gain (KPIG) to improve LLMs' instruction-following and generalization abilities.
© 2024 by Linnk AI