Large Language Model based Long-tail Query Rewriting in Taobao Search: Bridging the Semantic Gap for Improved E-commerce Search

Core Concepts

BEQUE is a comprehensive framework designed to bridge the semantic gap in long-tail queries, enhancing e-commerce search results.

Abstract

In the realm of e-commerce search, semantic matching is crucial for user experience and revenue. Existing query rewriting methods struggle with long-tail queries and "few-recall" issues. BEQUE addresses this by fine-tuning large language models (LLMs) and using contrastive learning to align with online objectives. Offline experiments show effectiveness in bridging the semantic gap, while online A/B tests reveal significant boosts in gross merchandise volume (GMV), number of transactions (#Trans), and unique visitors (UV) for long-tail queries on Taobao.
Key points:

Semantic matching importance in e-commerce search.
Challenges with existing query rewriting methods for long-tail queries.
BEQUE framework overview: multi-instruction supervised fine tuning, offline feedback, objective alignment.
Contributions of BEQUE: analysis of long-tail queries, three-stage fine-tuned framework, offline and online experiment results.

Stats

Offline experiments prove effectiveness of method in bridging semantic gap.
Online A/B tests reveal significant boosts in GMV, #Trans, and UV for long-tail queries.

Quotes

"In this paper, we present BEQUE, a comprehensive framework that Bridges the sEmantic gap for long-tail QUEries." - Authors
"BEQUE has been deployed on Taobao since October 2023." - Authors
"Our method can significantly boost gross merchandise volume (GMV), number of transaction (#Trans) and unique visitor (UV) for long-tail queries." - Authors
"We propose BEQUE to address the issue of semantic gap in long-tail queries." - Authors
"The main contributions of this work are listed as follows..." - Authors
"We introduce PRO to align the model’s objectives with those of Taobao." - Authors
"Through multiple experiments, we have demonstrated the effectiveness of our approach..." - Authors

Key Insights Distilled From

Large Language Model based Long-tail Query Rewriting in Taobao Search

by Wenjun Peng,... at arxiv.org 03-05-2024

https://arxiv.org/pdf/2311.03758.pdf

Deeper Inquiries

How does BEQUE compare to other query rewriting frameworks outside of e-commerce

BEQUE stands out from other query rewriting frameworks outside of e-commerce due to its specialized focus on addressing the semantic gap in long-tail queries within the e-commerce domain. While traditional query rewriting methods may struggle with optimizing long-tail queries and improving retrieval results, BEQUE leverages rejection sampling, auxiliary task mixing, and a multi-instruction supervised fine-tuning approach to enhance the model's understanding of e-commerce queries. This tailored approach allows BEQUE to generate rewrites that align with the objectives of Taobao search, leading to significant improvements in relevance, increment, hitrate metrics both offline and online.

What potential biases or limitations could arise from using reinforcement learning-based models like RL

Using reinforcement learning-based models like RL for query rewriting can introduce potential biases and limitations. One limitation is the reliance on an accurate reward model for training the RL algorithm. If the reward model does not accurately capture all aspects of desired behavior or if it introduces biases based on how rewards are defined or calculated, it can lead to suboptimal performance of the RL-based model. Additionally, RL models require extensive computational resources and time for training due to their iterative nature, which can be a limiting factor in real-time applications such as e-commerce search where low latency is crucial.

How might advancements in large language models impact future developments in e-commerce search technology

Advancements in large language models (LLMs) are poised to have a significant impact on future developments in e-commerce search technology. With their enhanced semantic understanding capabilities and ability to generate high-quality rewrites for complex long-tail queries, LLMs like those used in BEQUE offer improved user experience by bridging semantic gaps between user intent and search results. These advancements enable more personalized and relevant product recommendations based on natural language input from users. Furthermore, LLMs open up possibilities for incorporating advanced techniques like contrastive learning and objective alignment into e-commerce search systems to optimize retrieval outcomes even further.