核心概念
The authors propose a cross-problem learning method to leverage pre-trained Transformer models for the Traveling Salesman Problem (TSP) to efficiently train neural heuristics for solving other complex Vehicle Routing Problems (VRPs).
摘要
The paper presents a cross-problem learning approach to assist the training of neural heuristics for solving different VRP variants. The key ideas are:
-
Modularization of Transformer architectures for complex VRPs: The backbone Transformer for TSP is used as the base, with additional lightweight modules to handle problem-specific features.
-
Pre-training the backbone Transformer on TSP: The authors first train the Transformer model to solve the basic TSP problem.
-
Fine-tuning for downstream VRPs: The pre-trained backbone Transformer is then fine-tuned for other VRP variants, either by fully fine-tuning the entire model or by only fine-tuning the lightweight problem-specific modules with small adapter networks.
The experiments show that the full fine-tuning approach significantly outperforms training the Transformer from scratch for each VRP. The adapter-based fine-tuning methods also achieve comparable performance to the from-scratch approach, while being much more parameter-efficient. The authors also demonstrate the versatility of their method by applying it to different Transformer-based neural network architectures.
統計資料
Existing neural heuristics often train a deep architecture from scratch for each specific vehicle routing problem (VRP), ignoring the transferable knowledge across different VRP variants.
The proposed cross-problem learning leverages the pre-trained backbone Transformer for the Traveling Salesman Problem (TSP) to assist the training of neural heuristics for other complex VRP variants.
The full fine-tuning approach achieves significantly better performance than training the Transformer from scratch for each VRP.
The adapter-based fine-tuning methods deliver comparable performance to the from-scratch approach, while being much more parameter-efficient.
引述
"As an early attempt, we propose cross-problem learning for solving VRPs. The pre-trained backbone Transformer for a basic VRP (i.e., TSP) is used to foster the training of Transformers for downstream complex VRPs through fine-tuning."
"We develop different fine-tuning methods for cross-problem learning, i.e., the full and adapter-based fine-tuning, by which we fine-tune the entire Transformers or only the lightweight problem-specific modules along with small adapter networks."
"We empirically testify that the knowledge learned in the Transformer for TSP is well transferred to aid in training neural heuristics for other VRPs. While full fine-tuning achieves better performance than Transformer trained from scratch, adapter-based fine-tuning methods attain comparable performance, with far fewer parameters to be trained and stored."