Direct Nash Optimization: A Scalable Algorithm for Aligning Large Language Models with General Preferences

Direct Nash Optimization (DNO) is a provable and scalable algorithm that optimizes large language models to align with general preferences, outperforming reward-based approaches and achieving state-of-the-art results.