Structured Initialization Strategy Boosts Data-Efficient Learning in Vision Transformers

Incorporating convolutional inductive bias through a structured initialization strategy can significantly improve the data-efficient learning of vision transformers on small-scale datasets, while maintaining their flexibility for large-scale applications.