Measuring the Hermeneutic Circle in Explainable AI: A Human-Free Test Based on Data Horizons, Executable Claims, Part–Whole Agreement, and Machine Readers
Year: 2026
Type: article
Abstract: Explanations of machine learning models are read by people but rarely evaluated with them, and automatic metrics score attribution vectors rather than what a reader receives. We operationalise the horizon, the hermeneutic circle, and the hermeneutics of suspicion as the Hermeneutic Circle Test (HCT), a human-free screen for explanations of tabular classifiers. HCT checks whether an explanation’s evidence lies inside a calibrated data horizon, whether its sentences survive on-horizon interventions, whether its account of a case agrees with the whole model and at which scale, and whether it helps machine readers learn the model. Over seven explainers, four datasets, and two model families (mean ± standard deviation over eight settings), the best explainer, KernelSHAP, verified 0.65 ± 0.15 of its claims, while random attributions verified 0.22 ± 0.08. On the same explanations, established faithfulness, remove-and-retrain, and disagreement metrics missed impossible evidence and narratives contradicting their own attributions, and sign-blind metrics scored sign-flipped explanations like the originals. Claim tests withstood scaffold attacks that adapt to the horizon and to the test itself; a scaffold that memorises the audited records weakened them and deceived every explainer, but part–whole agreement still flagged it. HCT is a reproducible screen to apply before user studies.
DOI: 10.3390/make8110381
Author: Şadi Evren Şeker
Institutions: The University of Texas at Dallas, Haliç University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $2,172
OpenAlex ID: W7234297902
