DIALECTBENCH is a groundbreaking benchmark that evaluates NLP systems on a wide range of language varieties, highlighting disparities in performance across standard and non-standard dialects.
The author proposes a novel approach to evaluate NLPre systems through benchmarking, addressing the shortcomings of existing evaluation methods. The focus is on creating a reliable and comprehensive system for ranking NLPre tools.