MediSwift: Efficient Sparse Pre-trained Biomedical Language Models

Sparse pre-training in biomedical language models enhances efficiency and accuracy, setting new benchmarks.