Mixture-of-Experts Training and Sparse Upcycling
Year: 2026
Type: article
Abstract: A technical note on training sparse Mixture-of-Experts (MoE) models, covering expert routing strategies and the sparse upcycling approach for converting dense checkpoints into MoE architectures.
Author: Santos, Dheiver
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7171300431
FWCI:
Cites:
Cited by:
Related to: