MediSwift introduces efficient sparse pre-training for biomedical language models, achieving significant reductions in training FLOPs while maintaining high performance on specialized tasks.
