Work

Precision vs. Pragmatics: Benchmarking Post-Training Quantization Trade-offs in Sovereign Korean Large Language Models

Year: 2026
Type: preprint
Abstract: The rapid deployment of Sovereign Large Language Models (LLMs) in South Korea requires balancing high computational efficiency with strict preservation of linguistic pragmatics, particularly hierarchical honorifics (Jondaetmal) and cultural context. While 4-bit Post-Training Quantization (PTQ) significantly reduces memory footprints for edge deployment on Neural Processing Units (NPUs), its impact on culturally sensitive Korean generation remains underexplored. In this study, we empirically benchmark the memory efficiency and pragmatic fidelity of 4-bit NormalFloat (NF4) quantization applied to a 2.8-billion-parameter Korean sovereign LLM (EEVE-Korean-Instruct-2.8B-v1.0) on an NVIDIA T4 GPU. Our results demonstrate that NF4 quantization reduces model memory footprint by 67.2% (from 5.64 GB in FP16 to 1.85 GB in INT4) while preserving grammatical honorific structure in formal workplace communication.
Language: English
Open Access status: green
OpenAlex ID: W7234110056
Cites:
Cited by:
Related to: 10