Mixture-of-Experts models
Find Mixture-of-Experts architectures, sparse activation, routing methods, and related open models.
Map the topic before choosing
For MoE, inspect total parameters, active parameters, routing, parallelism, and inference memory together. This page keeps architectures, mechanisms, and models on one exploration path.
Use this page to understand the topic and discover candidates. When you are ready to compare deployment, budget, or capability constraints, open the decision guide above.
Curated KG entry points
A small set of entry points for discovering this topic, not a paid ranking or an official product review. Return to the homepage for the full semantic retrieval and second-stage summary.
MoE
Model architecture with sparsely activated expert networks, feature of Seed1.6.
MechanismMixture-of-Experts
Makes local frontier inference arithmetically feasible via sparsity.
MechanismMoE Routing
KG-recorded mechanism related to MoE Routing; detailed English notes are not yet available.
ModelPhi-mini-MoE-instruct
Phi-mini-MoE is a lightweight Mixture of Experts (MoE) model with 7.6B total parameters and 2.4B activated parameters. It is compressed and distilled from the base model shared by Phi-3.5-MoE and GRIN-MoE using the SlimMoE approach, then post-trained via supervised fine-tuning and direct preference optimization for instruction following and safety. The model is trained on Phi-3 synthetic data and filtered public documents, with a focus on high-quality, reasoning-dense content. It is part of the SlimMoE series, whic
ModelPhi-tiny-MoE-instruct
Phi-tiny-MoE is a lightweight Mixture of Experts (MoE) model with 3.8B total parameters and 1.1B activated parameters. It is compressed and distilled from the base model shared by Phi-3.5-MoE and GRIN-MoE using the SlimMoE approach, then post-trained via supervised fine-tuning and direct preference optimization for instruction following and safety. The model is trained on Phi-3 synthetic data and filtered public documents, with a focus on high-quality, reasoning-dense content. It is part of the SlimMoE series, whic
MechanismCursor Router
KG-recorded mechanism related to Cursor Router; detailed English notes are not yet available.
ModelQwen1.5-MoE-A2.7B-Chat
Qwen1.5-MoE is a transformer-based MoE decoder-only language model pretrained on a large amount of data. For more details, please refer to our blog post and GitHub repo.
Modeldeepseek-moe-16b-base
1. Introduction to DeepSeekMoE See the Introduction for more details. 2. How to Use Here give some examples of how to use our model. #### Text Completion
Share feedback on this topic
Anonymous feedback is attached to this topic page, stays outside the KG, and is visible to other visitors.