Impossible Distillation for Paraphrasing and Summarization: A Novel Framework
Distilling high-quality paraphrase datasets and models from low-quality LMs is achievable through IMPOSSIBLE DISTILLATION, leveraging paraphrastic proximity and critic-guided distillation.
