Real-Time Hallucination Detection in Resource-Constrained LLMs via Logit-Level Shannon Entropy Mapping
Year: 2026
Type: preprint
Abstract: Deploying Large Language Models (LLMs) on resource-constrained edge and consumer hardware presents severe reliability challenges due to the computational infeasibility of secondary evaluator models and multi-sample consensus mechanisms for hallucination mitigation. While state-of-the-art frameworks such as Semantic Entropy and SelfCheckGPT achieve strong detection accuracy in cloud environments, their reliance on K-fold stochastic generation and auxiliary Natural Language Inference (NLI) models multiplies memory bandwidth and latency by 5× to 10×, exceeding the strict memory envelopes of edge devices. This paper introduces a lightweight, single-pass telemetry framework that intercepts pre-softmax logit distributions during the auto-regressive forward pass to compute token-level Shannon Entropy (H(X)) in real time. Utilizing a custom C#/.NET visual diagnostic architecture (Forensic Canvas) interfaced via native P/Invoke hooks with a 4-bit quantized 8-billion-parameter model (Meta-Llama-3-8B-Instruct, Q4_K_M) executing entirely within an 8 GB RAM CPU environment, we demonstrate that structural and semantic hallucinations are consistently preceded by deterministic spikes in vocabulary probability dispersion. Empirical evaluations across 200 factual question-answering prompts derived from TruthfulQA and HaluEval benchmarks reveal that our single-pass logit entropy monitor achieves an Area Under the Receiver Operating Characteristic curve (AUROC) of 0.819, outperforming single-pass Maximum Softmax Probability (0.682) and Sequence Perplexity (0.714) while matching multi-pass SelfCheckGPT (0.815) at only 1.2 ms/token latency overhead and < 15 MB additional memory. Furthermore, we propose a deterministic “Entropy Kill-Switch” (τH = 4.5 bits) that proactively halts hallucinatory inference streams prior to factual divergence, reducing wasted auto-regressive compute cycles by 41.8% on power-limited edge hardware.
Author: Munir, Badar
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225810214
Cites:
Cited by:
Related to: