Precision vs. Pragmatics: Benchmarking Post-Training Quantization Trade-offs in Sovereign Korean Large Language Models
Year: 2026
Type: preprint
Abstract: The rapid deployment of Sovereign Large Language Models (LLMs) in South Korea requires balancing high computational efficiency with strict preservation of linguistic pragmatics, particularly hierarchical honorifics (Jondaetmal) and cultural context. While 4-bit Post-Training Quantization (PTQ) significantly reduces memory footprints for edge deployment on Neural Processing Units (NPUs), its impact on culturally sensitive Korean generation remains underexplored. In this study, we empirically benchmark the memory efficiency and pragmatic fidelity of 4-bit NormalFloat (NF4) quantization applied to a 2.8-billion-parameter Korean sovereign LLM (EEVE-Korean-Instruct-2.8B-v1.0) on an NVIDIA T4 GPU. Our results demonstrate that NF4 quantization reduces model memory footprint by 67.2% (from 5.64 GB in FP16 to 1.85 GB in INT4) while preserving grammatical honorific structure in formal workplace communication.
Author: Maliye, Radhika
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7234110056
