Efficient Approximate k-NN Search with Cross-Encoders through Sparse Matrix Factorization

This paper proposes an efficient approach to perform approximate k-nearest neighbor (k-NN) search with cross-encoder models by learning a low-dimensional embedding space that approximates the cross-encoder scores. The key innovations are: (1) a sparse matrix factorization method to compute item embeddings that align with the cross-encoder, and (2) an adaptive test-time retrieval method that incrementally refines the test query embedding to improve the approximation of cross-encoder scores.