Rerankers Often Decrease Information Retrieval Quality When Scoring More Documents

Contrary to common assumptions, using rerankers to score a larger number of documents in information retrieval often leads to decreased performance, even falling below the accuracy of simpler retrieval methods.