Nyonic's Innovative 7B Language Model: Advancements in Multilingual Capabilities and Efficient Training

Nyonic has developed a 7B parameter language model, Wonton 7B, with novel techniques such as an Online Data Scheduler, Rotary Positional Embeddings, and QK-LayerNorm to enhance stability and performance. The model demonstrates competitive results on various multilingual and English benchmarks.