Dynamic Reversible Dual-Residual Networks for Memory-Efficient Finetuning of Pretrained Models

Dr2Net, a novel family of reversible network architectures, enables finetuning of pretrained models with substantially reduced memory consumption while preserving accuracy.