Remembering Transformer: A Continual Learning Approach Inspired by Complementary Learning Systems

Remembering Transformer employs a mixture-of-adapters and a generative model-based routing mechanism to alleviate catastrophic forgetting in continual learning by dynamically routing task data to relevant adapters.