When Does GNR-Q Complement Low-Rank Quantization Correction? Evidence Across Precisions and Model Families
Year: 2026
Type: preprint
Abstract: Guided Neural Reconstruction for Quantization (GNR-Q) is an inference-time, state-conditioned correction framework introduced and initially validated in prior methodological work (Baeck et al., 2026). This follow-up study does not introduce GNR-Q anew; it investigates the conditions under which a frozen GNR-Q sidecar improves representation fidelity and next-token prediction across quantization precisions, correction methods, and model families. In a 27B model family, Study 2-1 reduced reference-relative hidden-state MSE across six quantization conditions (2.24–6.55%) using a roughly 3.94-MB sidecar, and Study 2-2 observed positive held-out cross-entropy (CE) improvements in all six conditions, four surviving Holm adjustment. Adding frozen GNR-Q to two custom low-rank controls yielded statistically supported incremental benefit in seven of twelve original comparisons. A separately protocol-locked, raw-text external validation extended the evaluation to Gemma 4 31B-IT and EXAONE 4.0 32B at EXL3 Q2/Q3/Q4. Equal-bit family-average standalone CE improvements were 0.2485 and 0.0166 nats/token, respectively, with larger Q2 gains than Q3 or Q4 in both families. Adding the frozen GNR-Q corrector after EoRA-style or QERA-approx controls further reduced family-average CE in four Holm-adjusted comparisons, although EXAONE Q4 combinations increased CE. These results characterize condition-dependent transfer and incremental compatibility of an existing reconstruction method; they do not establish universal improvement, matched-budget superiority, or superadditive synergy. Relationship to Foundational Research (Study 1) This manuscript is a follow-up empirical investigation building upon the foundational GNR-Q study: Baeck, S., Pyo, D., & Suk, H. (2026). GNR-Q: Inference-Time State-Conditioned Reconstruction for Memory-Efficient Quantized Language Models. Zenodo, Version 2. https://doi.org/10.5281/zenodo.23113127 The foundational study introduced the GNR-Q framework and evaluated its initial feasibility using Qwen3-4B, including controlled low-bit reconstruction and packed W4 deployment. The present study extends that methodological foundation by investigating six publicly available quantization conditions of a 27B model, incremental correction effects after EoRA-style and QERA-approx controls, and external validation across the Gemma 4 and EXAONE 4.0 model families. The original method formulation and initial validation are attributed to Study 1. The distinct contribution of Study 2 lies in its comparative evaluation, analysis of condition-dependent effects, and cross-family validation. Publication Status This manuscript is an unreviewed preprint and should not be interpreted as an accepted journal publication. The foundational Study 1 manuscript has been submitted to Transactions on Machine Learning Research (TMLR) and has not yet been accepted, as of November 2026. A revised version of this follow-up manuscript is planned for journal submission after completion of the Study 1 review process. Supplementary Materials The accompanying archive contains model and dataset provenance records, verification manifests, and archived numerical audit summaries supporting the reported results. It does not constitute a complete release of all original GPU execution traces or model checkpoints.
Author: Anonymous, Authors
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7234062179
