AI background and advice uptake after low practice performance: A reproduction and exploratory extension of Horowitz and Kahn
Year: 2026
Type: preprint
Abstract: Horowitz and Kahn (2024a) report a nonlinear association between AI background and answer switching after AI advice in a national-security-framed experiment. This study reproduces the focal background coefficients in their Table 2 model 7 to published precision and extends the analysis by separating opportunities to adopt correct and incorrect advice. The exploratory extension uses 18,560 AI-advice trials from 4,191 respondents who answered fewer than three of five practice questions correctly. At an AI-background index of 0.6 versus the respondent median, the primary quadratic model estimates 8.75 percentage points less beneficial-advice adoption and 10.93 points less harmful adoption. The beneficial estimate becomes less precise when the original covariates are restored, while harmful-adoption intervals remain below zero across those specifications. The adjusted net-accuracy comparison is inconclusive. Respondents with background of at least 0.6 nevertheless improve their observed accuracy by 3.74 points from initial to final answers. High-background support is sparse, all extension analyses are post hoc, and the historical log's cluster count remains unresolved. These results preserve the selected original association while qualifying what it establishes about the quality of advice use. They identify neither a causal effect of AI education nor general psychological calibration. This record contains the preprint and technical supplement for a selected-result reproduction and exploratory extension of Horowitz and Kahn (2024). The accompanying software and aggregate results are deposited separately. The original respondent data and authors' R/Stata scripts remain at the version 1.0 Harvard Dataverse deposit. The extension was not preregistered, and the manuscript distinguishes internal verification from independent CODECHECK certification.
Author: Phua, Dennis
Language: English
Subfield: General Decision Sciences
Field: Decision Sciences
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7233554430
Cites:
Cited by:
Related to:
