Uncovering the Internal Numerical Representations in ALBERT Language Models

Transformer-based language models like ALBERT exhibit an innate understanding of numerical concepts, as reflected in the internal representations of numerical tokens.