Reverse KLD is proposed for distilling LLM knowledge into smaller models, improving performance and reducing exposure bias.
