Pare4Bit: Quantization, Depth Pruning, and Model Selection for the WMT26 Model Compression Task
Year: 2026
Type: conference-paper
Abstract: We describe Team Pare4Bit's submissions to the WMT26 Model Compression shared task on the three evaluation pairs Czech→German, English→Simplified Chinese, and English→Egyptian Arabic.In the constrained track we compress google/gemma-3-12b-it with a data-light pipeline-vision-tower removal followed by GPTQ W4A16 quantization-reaching 43.2 mean chrF at 7.6 GB, 3.1× smaller than the bf16 text-only reference for a 0.3 chrF drop.We further explore depth pruning with LoRA knowledge-distillation healing, which recovers a collapsed pruned model to within 1.3 chrF of the INT4 primary and adds an engine-agnostic speedup.In the unconstrained track we treat model selection as a first-class compression decision: the newer Gemma-4-12B, deployed via quantization-aware-trained INT4, outperforms the constrained Gemma-3 baseline by +2.5 mean chrF (+4.7 on Egyptian Arabic, where it produces the dialect rather than Modern Standard Arabic).We report several negative results-activation quantization instability on Gemma-3, re-segmentation under naive vocabulary pruning, and a chrF-versus-COMET disagreement on a sentence-level MT specialist applied to paragraphs-as practical guidance on what does and does not compress this model family.
Author: Nail Fakhrutdinov
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: bronze
OpenAlex ID: W7224645072