ConsistentEE: A Reinforcement Learning-Based Early Exiting Method for Efficient Language Model Inference

ConsistentEE formulates the early exiting process as a reinforcement learning problem, where a policy network decides whether to exit or continue at each intermediate layer. The training objective only requires each instance to be predicted correctly by one internal classifier, in contrast to existing methods that impose all internal classifiers to predict all instances correctly.