Year: 2026
Type: preprint
Abstract: Capture, tracing and coding data for an audit of what the citations displayed by four generative search products actually reach. Six questions about Virginia data-center policy were put to ChatGPT, Gemini, Perplexity and Grok, on their free tiers, each in the mode the product labels Temporary chat, Incognito or Private, twice each, inside a 48-hour window on 12 and 13 September 2026. The authoritative record for each question, its date and its denominator were entered before any prompt was issued. Across the 48 answers the products displayed 185 citations, which reached 144 distinct terminal pages. Nineteen were the record itself. None documented an independent check. Ninety-two could not be followed from the interface, 77 of them because the product displayed a title with no link. Contents. The capture and tracing workbook (11 sheets: pre-registration, controlled vocabulary with post-registration additions marked, terminals, citations, captures, summary, sensitivity, adjudication, page evidence, blind second coder). A validator script that enforces the vocabulary and the internal joins. Structured capture records for all 48 answers. Full recovered answer text for both Grok runs. Verbatim page evidence for every unique cited URL. Both coders' codes. Local copies of the six pre-registered authoritative records. Method and limits. Tracing, page retrieval and first coding were performed by an AI assistant under the author's direction; a second AI instance coded the 58 unique cited URLs blind and agreed with the first on 42 of the 52 it could code. The author ruled on the contested codings and did not adjudicate a random subsample, so no representative agreement rate is reported. Five vocabulary values were added after pre-registration and are marked as such. The scope is six questions in one policy domain, four products on free tiers, two runs, one window; nothing here is a claim about the web generally or about pages the products did not display. Every headline count is reported under each of the alternative codings that move it. The README lists the five disclosures that touch a reported number. Companion to the essay "A dozen links may still amount to one source" (SSRN). Files under records/ are public documents of Loudoun County, James City County, the Virginia Joint Legislative Audit and Review Commission, the U.S. Securities and Exchange Commission (Dominion Energy Form 10-K) and the U.S. Energy Information Administration, included as local copies so the coded figures can be checked. The CC BY 4.0 licence applies to the author's own work in this deposit.
Source: arXiv (Cornell University)
Authors: Liu, Zhongxuan, Zhou, Sicheng, Wang, Hongzhi
Institution: Harbin Institute of Technology
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7212075193