Enabling Small Language Models to Perform Step-by-Step Reasoning through Symbolic Chain-of-Thought Distillation

Smaller language models can be trained to perform step-by-step reasoning through a process called Symbolic Chain-of-Thought Distillation, where they learn to generate coherent and effective chain-of-thoughts by distilling from a larger teacher model.