Sign In
insight - Large Language Model Distillation
Distilling Large Language Models for Reasoning: Focusing on Decomposition for Efficiency and Generalization
Distilling the decomposition capability of large language models (LLMs) offers a cost-effective and generalizable approach to improve reasoning in smaller models, while distilling the solving capability proves less effective and less generalizable.
Distilling Instruction-Following Abilities of Large Language Models Using Task-Aware Curriculum Planning
Strategically distilling knowledge from powerful LLMs into smaller models using task-aware curriculum planning and response refinement significantly improves the instruction-following abilities of smaller LLMs, even surpassing larger models trained on more extensive datasets.
Efficient Distillation of Large Language Models Using Bi-directional Logits Difference Loss
The Bi-directional Logits Difference (BiLD) loss effectively distills large language models by filtering out long-tail noise in logits and leveraging the internal ranking information of logits.
Products
© 2024 by Linnk AI