JetMoE-8B: Achieving Llama2 Performance with Only $0.1 Million

JetMoE-8B, a new 8B-parameter Large Language Model (LLM), demonstrates impressive performance while being trained with less than $0.1 million, outperforming the larger Llama2-7B and Llama2-13B-Chat models.