Understanding how preference distinguishability impacts the learning dynamics of language models aligned with human feedback.