Residual-based large language models (LLMs) can significantly enhance performance in biomedical imaging tasks, setting new benchmarks in 2D and 3D classification.