The paper introduces the Retrieval-Oriented Knowledge (ROK) framework to address the inference inefficiency problem of sample-level retrieval-based click-through rate (CTR) prediction models.
The key highlights are:
ROK constructs a knowledge base that imitates the aggregated representations from a pre-trained sample-level retrieval-based model (e.g., RIM) using a decomposition-reconstruction paradigm. This allows efficient inference by replacing the time-consuming retrieval process with a simple forward propagation of the neural network.
ROK utilizes knowledge distillation and contrastive learning to optimize the knowledge base, enabling the integration of retrieval-enhanced representations with various backbone CTR models in both instance-wise and feature-wise manners.
Extensive experiments on three large-scale datasets show that ROK achieves competitive performance compared to existing retrieval-based CTR methods while maintaining superior inference efficiency. ROK also enhances the performance of various backbone CTR models due to its exceptional compatibility.
The neural knowledge model in ROK serves as a compact surrogate for the retrieval pool, making sample-level retrieval-based methods feasible for industrial applications, which were previously deemed impractical due to inference inefficiency.
toiselle kielelle
lähdeaineistosta
arxiv.org
Tärkeimmät oivallukset
by Huanshuo Liu... klo arxiv.org 04-30-2024
https://arxiv.org/pdf/2404.18304.pdfSyvällisempiä Kysymyksiä