Greed is All You Need: An Evaluation of Tokenizer Inference Methods

Greedy inference methods perform well for tokenizers, with SaGe outperforming others in morphological alignment.