
Incorporating margin values into the training process significantly improves the effectiveness of reward models in capturing human preferences.


coremsg

enhancing-reward-model-performance-by-incorporating-preference-margin


Enhancing Reward Model Performance by Incorporating Preference Margin


title_rewrite


TINYLLM proposes a novel knowledge distillation paradigm that learns a small student language model by distilling reasoning capabilities from multiple large teacher language models, enabling the student to outperform the teachers while using significantly fewer parameters.


tinyllm-distilling-diverse-reasoning-capabilities-from-multiple-large-language-models-into-a-compact-student-model


TINYLLM: Distilling Diverse Reasoning Capabilities from Multiple Large Language Models into a Compact Student Model



CroissantLLM is a 1.3B parameter language model pre-trained on a balanced corpus of 1.5T English and French tokens, designed to provide high-performance and resource-efficient bilingual capabilities.


introducing-croissantllm-a-highly-capable-and-transparent-bilingual-language-model


Introducing CroissantLLM: A Highly Capable and Transparent Bilingual Language Model
