The paper introduces Block-Max Pruning (BMP), a novel query processing strategy optimized for indexes generated by learned sparse retrieval models. BMP employs a block filtering mechanism to prioritize clusters of documents based on their potential relevance, using an optimized computation of range-based upper bounds. It evaluates promising subsets of documents through a hybrid between inverted and forward index structures.
The key highlights and insights are:
Learned sparse retrieval models, such as SPLADE, ESPLADE, and uniCOIL, exhibit structural variations in query and document statistics compared to traditional retrieval models, leading to performance discrepancies with existing query optimization techniques.
BMP substantially outperforms existing dynamic pruning strategies like MaxScore, BlockMaxWand, Anytime, and Clipping, offering 2.9x to 7.5x faster query processing times for safe retrieval on the SPLADE model.
For approximate retrieval, BMP achieves the best trade-off between efficiency and effectiveness, with sub-millisecond average response times and negligible loss in precision compared to exhaustive search.
BMP's efficiency is further improved by up to 2.5x when using a raw block-max index, without compression, demonstrating the effectiveness of the block-based pruning approach.
The paper also explores query term pruning as an additional approximation mechanism within the BMP framework, achieving sub-millisecond retrieval times with only a slight decrease in precision.
Overall, the proposed BMP strategy represents a significant advancement in optimizing learned sparse retrieval, addressing the efficiency challenges and enabling faster and more effective query processing.
To Another Language
from source content
arxiv.org
Key Insights Distilled From
by Antonio Mall... at arxiv.org 05-03-2024
https://arxiv.org/pdf/2405.01117.pdfDeeper Inquiries