Year: 2026
Type: article
Abstract: ABSTRACT Despite significant advances since 2023 in large language model (LLM) capabilities and the introduction of web‐enabled search functionality, citation hallucination remains a persistent problem in AI‐generated academic content. We test GPT‐4o, DeepSeek V3, web‐enabled GPT‐4o, and web‐enabled GPT‐5.1 across prompts derived from Journal of Economic Literature categories. Using a hierarchical citation validation framework, we find that even an advanced web‐enabled model produces non‐existent citations. While web‐enabled models show improvement over their predecessors, the error rates remain high for scholarly applications. Our findings suggest that the fundamental challenge of citation hallucination persists despite technological advances, with implications for academic integrity and the reliable use of AI in research contexts. We provide an online tool for researchers to run LLMs through the benchmark test using our prompts.
Source: Southern Economic Journal
DOI: 10.1002/soej.70069
Authors: Joy Buchanan, Stephen Hill, Olga Shapoval
Institution: Samford University
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: bronze
OpenAlex ID: W7220913748