Impact of Position Bias on Language Models in Token Classification Tasks

Language models trained on datasets with skewed position distribution of classes or tags can suffer from position bias, leading to performance degradation on tokens appearing in out-of-distribution positions within sequences.