SAFER-INSTRUCT: Automated Generation of Large-Scale Preference Data for Aligning Language Models

SAFER-INSTRUCT introduces a novel pipeline for efficiently constructing large-scale preference data without human annotators, enabling the development of safer and more capable AI systems.