Efficient and Scalable Mixture-of-Experts Inference through Algorithm-System Co-Design

An algorithm-system co-design called Pre-gated MoE that enables fast and memory-efficient deployment of Mixture-of-Experts (MoE) based large language models.