Bibliographic Information: Redjdal, A., Pinto, L., & Desmarais, M. (2024). Optimizing Encoder-Only Transformers for Session-Based Recommendation Systems. arXiv preprint arXiv:2410.11150v1.
Research Objective: This paper investigates the effectiveness of encoder-only transformer models, specifically BERT and DeBERTa, for session-based recommendation systems (SBR) and introduces a novel masking technique called Sequential Masked Modeling (SMM) to enhance their performance.
Methodology: The authors propose SMM, which combines data augmentation through window sliding with a penultimate token masking strategy, to improve the model's ability to capture sequential dependencies in session data. They further optimize the transformer architectures by implementing weight tying, pre-layer normalization, and contextual positional encoding. The performance of the optimized models is evaluated on three benchmark datasets: Yoochoose 1/64, Diginetica, and Tmall, using Precision@20 and MRR@20 as evaluation metrics.
Key Findings: The results demonstrate that the proposed BERT-SMM and DeBERTa-SMM models consistently outperform traditional single-session approaches across all three datasets. Notably, BERT-SMM achieves the best overall performance, even surpassing some state-of-the-art cross-session and multi-relation methods that utilize more extensive user data.
Main Conclusions: The study highlights the potential of encoder-only transformers in session-based recommendation tasks, particularly when enhanced by the SMM technique. The authors conclude that SMM effectively captures sequential dependencies and improves recommendation accuracy, even when limited to single-session data.
Significance: This research contributes to the advancement of SBR by demonstrating the efficacy of optimized encoder-only transformer models. The proposed SMM technique offers a promising approach to enhance performance in recommendation systems that rely on session-based data.
Limitations and Future Research: While the study focuses on single-session recommendations, future research could explore incorporating cross-session or multi-relational information to further improve performance. Additionally, investigating the applicability of SMM in other recommendation domains would be beneficial.
To Another Language
from source content
arxiv.org
Key Insights Distilled From
by Anis Redjdal... at arxiv.org 10-16-2024
https://arxiv.org/pdf/2410.11150.pdfDeeper Inquiries