Rotary Position Embedding for Vision Transformer: Comprehensive Analysis and Performance Evaluation

RoPE demonstrates impressive extrapolation performance in Vision Transformers, leading to improved backbone performance.