An Empirical Analysis of Factual Errors in Human-Written Text and Its Application to Factual Error Detection
Year: 2026
Type: conference-paper
Abstract: Factual Error Detection (FED), which is the task of identifying factually incorrect spans in a given text, has long been recognized as an important research problem.However, with the rapid rise of large language models (LLMs), research attention has shifted toward factual errors specific to LLM-generated text (hallucinations) and their detection.As a result, the detection of factual errors in human-written text has been relatively neglected.To address this gap, we first distill a taxonomy of humaninduced factual errors by analyzing corrections of newspaper articles, a representative source of text that is guaranteed to be human-written and contains few grammatical errors.Our analysis revealed that there are characteristic categories such as kanji misconversions and unit errors, which are not focused in existing hallucination benchmarks.Based on the taxonomy, we then evaluate the FED capability of vanilla LLMs on synthesized realistic test cases and real corrections.Experimental results demonstrated that even high-performance LLMs such as GPT-5.4achieved only word-level F1 score of 52% on the synthetic evaluation data, highlighting the task difficulty.Furthermore, a detailed analysis by detection difficulty revealed the current state of FED.* This work was conducted while the author was with Nikkei Inc.1 Proofreading might also address factual errors, but we distinguish between proofreading and revising in this sense.
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: bronze
OpenAlex ID: W7166555405