MeRino: Entropy-Driven Design for Efficient Language Models on IoT Devices

The author proposes an entropy-driven framework to design mobile-friendly generative language models, optimizing network architecture by maximizing transformer decoder entropy and model trainability under computational budgets.