Leveraging Intra-Layer Representations for Rehearsal-Free Continual Learning with Pre-Trained Models
Leveraging intra-layer representations and second-order feature statistics from pre-trained models can enhance performance and robustness in continual learning settings, without requiring access to past data.
Continual Learning with Pre-Trained Models: A Comprehensive Survey of Recent Advancements
Continual learning with pre-trained models has emerged as a promising approach to overcome the challenge of catastrophic forgetting, leveraging the strong generalization capabilities of pre-trained models.
Continual Learning with Pre-trained Models: Bridging Theory and Practice through Truncated Singular Value Decomposition
This work proposes ICL-TSVD, a method that bridges the gap between empirical performance and theoretical guarantees in continual learning with pre-trained models. ICL-TSVD integrates the strengths of RanPAC into the Ideal Continual Learner framework and addresses the ill-conditioning of lifted features through continual SVD truncation, achieving both stability and strong performance.
