Utilizing Pre-trained Image Transformers in Federated Split Learning improves model robustness and training efficiency.