Improving Class Incremental Learning by Mimicking the Oracle Model Representations at the Initial Phase

Mimicking the representations of the oracle model, which is trained on all classes, at the initial phase of class incremental learning can significantly boost the overall performance.