Efficient Pretraining of Masked Autoencoders in a Single Day

This work proposes efficient training recipes for Masked Image Modeling (MIM) based self-supervised learning, focusing on mitigating data loading bottlenecks and employing progressive training techniques to significantly reduce pretraining time while maintaining high performance.