Transformer-based language models like ALBERT exhibit an innate understanding of numerical concepts, as reflected in the internal representations of numerical tokens.