Concetti Chiave
Optimizing NTK convergence and generalization enhances FSCIL performance.
Sintesi
The content delves into the optimization of Neural Tangent Kernel (NTK) in Few-Shot Class-Incremental Learning (FSCIL). It explores the impact of network width, self-supervised pre-training, logit-label alignment, and logits diversity on FSCIL performance. The study emphasizes the importance of NTK convergence and generalization for improved results.
Network Width Experimentation:
Widening convolutional layers improves FSCIL performance.
ResNet architectures show consistent enhancement with increased width.
Self-Supervised Pre-Training Impact:
SparK outperforms contrastive learning methods.
Generative strategies like SparK show better results than contrastive learning approaches.
Logit-Label Alignment Analysis:
Margin-based losses, especially curricular alignment, enhance model generalization.
Curricular alignment balances easy and hard samples for improved performance.
Logits Diversity Exploration:
Different mixup mechanisms affect logits diversity and model performance.
Transformer models show promise in closing the efficiency gap with ConvNets.
Experimental Setup:
Evaluation conducted on CIFAR100, CUB200-2011, miniImageNet, and ImageNet100 datasets.
Methodologies Evaluated:
CEC [4], ALICE [29], DINO [38], MAE [41], SparK [44], MoCo-v3 [42], simCLR [39], BYOL [43] evaluated for their impact on FSCIL performance.
Statistiche
On popular FSCIL benchmark datasets, NTK-FSCIL elevates end-session accuracy by 2.9% to 8.7%.
Citazioni
"Our network acquires robust NTK properties, significantly enhancing its foundational generalization."
"Incorporating NTK into FSCIL presents the challenge of ensuring that a finite-width network exhibits NTK properties akin to those of an infinitely wide network."