Not All Linguistic Capabilities Degrade Equally Under Quantization
Year: 2026
Type: preprint
Abstract: Low-precision quantization enables efficient LLM deployment, but its effect on distinct linguistic capabilities is poorly characterised: prior evaluations report aggregate metrics (perplexity, task accuracy), obscuring differential degradation. We evaluate four instruction-tuned models (Qwen 2.5-7B, Mistral 7B, Llama 3.1-8B, Gemma-2-9B) across four NLG tasks (summarisation, dialogue, data-to-text, simplification) and five quantization conditions (4-bit AWQ/GPTQ/GGUF, 2-bit GGUF, 16-bit baseline), measuring lexical accuracy (ROUGE), syntactic well-formedness (grammar error rate), factual consistency (AlignScore), and discourse/instruction-following (GPT-4o-mini judge). Across 92 conditions (≈500 samples each): (1) Friedman tests reject uniform degradation in 84% of model-task-bit-width combinations (p < 0.05); (2) factual consistency is more stable than grammar at 4-bit across all architectures (AlignScore mean change: −1.8%, d = −0.355; grammar degrades more in 18/27 cells, Bonferroni-corrected); (3) grammar direction is architecture-dependent: Qwen degrades (+19.0%) while Llama improves (−12.8%); (4) grammar improvement on dialogue (3/4 models) is not a length artefact: per-sample length–grammar correlations are near zero, and Mistral falsifies the length hypothesis (output +34.4% longer, grammar rate −37.9%). Compression does not yield a single robustness profile; it induces capability-specific changes depending on architecture, task, and quantization method.
Author: Majumdar, Arunabh
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7204124732
Cites:
Cited by:
Related to: