LIBMoE: A Library for comprehensive benchmarking Mixture of...

What is Mixture of Experts?

Multi-Head Mixture-of-Experts

Jamba: A Hybrid Transformer-Mamba Language Model

Mixtral of experts

Hinton vs LeCun vs Ng vs Tegmark vs O

CS25 I Stanford Seminar - Mixture of Experts (MoE) paradigm and the Switch Transformer