Beyond Retraining-Free MoE Compression: A Cost-Normalized Study of Post-Compression Adjustment
Year: 2026
Type: conference-paper
Abstract: Retraining-free MoE compression reduces deployment memory by pruning or merging experts, but often treats the compressed checkpoint as the final artifact.We argue that this view is incomplete: compressed MoE checkpoints are better understood as compressed initializations that benefit from a tiny postcompression adjustment stage.Across two MoE LLM backbones, four pruning/merging methods, three expert-retention ratios, and 28 benchmarks, we compare LM fine-tuning and teacher-based KD under matched small-data budgets and measured GPU costs.Using only 3,000 C4 examples and a single epoch of adjustment, FULL FT recovers 37.3% of the original-to-compressed performance gap on average.Moreover, LM fine-tuning is more costeffective than standard token-level KD, and fullparameter adjustment gives the strongest costrecovery trade-off among the tested scopes.These results suggest that retraining-free compression should be paired with small postcompression adjustment to recover a substantial portion of the performance lost during compression.
Authors: Sieun Hyeon, Jaeyoung Do
Institution: Seoul National University
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: bronze
OpenAlex ID: W7212002995