Основные понятия
The author introduces a new semantic-token paradigm and proposes a discrete semantic tokenization approach, UIST, for user and item representation. UIST facilitates swift training and inference while maintaining a conservative memory footprint.
Аннотация
In the quest to enhance click-through rate (CTR) prediction models by incorporating item content information efficiently, the author presents a novel approach called UIST. This method quantizes dense embedding vectors into discrete tokens with shorter lengths, offering significant space compression while maintaining efficiency in training and inference. By introducing a semantic-token paradigm, the author addresses the challenge of integrating item content into CTR prediction models within industrial constraints effectively.
The paper discusses various paradigms used for CTR prediction, comparing their efficiency and memory consumption. It highlights the advantages of UIST over other approaches in terms of space compression and accuracy. The proposed hierarchical mixture inference module dynamically adjusts the significance of user-item interactions at different levels of granularity, enhancing the overall performance of CTR prediction models.
Through experiments on a real-world news recommendation dataset, MIND, the effectiveness of UIST is validated against modern deep CTR models like DCN, DeepFM, and FinalMLP. The results demonstrate that UIST achieves substantial memory compression while maintaining high accuracy compared to existing paradigms. The study encourages further exploration of semantic tokenization's potential in boosting recommendation efficiency across diverse applications.
Статистика
Our approach offers about 200-fold space compression.
We set the number of transformer layers to 6.
During semantic tokenization, we set the residual depth to 4.
The codebook size for each layer is 64.
Цитаты
"Incorporating item content information into CTR prediction models remains a challenge."
"Our experimental results showcase the effectiveness and efficiency of UIST for CTR prediction."
"UIST greatly reduces space consumption while maintaining time efficiency."