Transferable Modularity in Modular Deep Learning: A Knowledge Distillation Case Study

Transferable modularity in modular deep learning can enhance knowledge transfer between models, even when they are incompatible, through the lens of knowledge distillation.