Work

Measuring the Hermeneutic Circle in Explainable AI: A Human-Free Test Based on Data Horizons, Executable Claims, Part–Whole Agreement, and Machine Readers

Year: 2026
Type: article
Abstract: Explanations of machine learning models are read by people but rarely evaluated with them, and automatic metrics score attribution vectors rather than what a reader receives. We operationalise the horizon, the hermeneutic circle, and the hermeneutics of suspicion as the Hermeneutic Circle Test (HCT), a human-free screen for explanations of tabular classifiers. HCT checks whether an explanation’s evidence lies inside a calibrated data horizon, whether its sentences survive on-horizon interventions, whether its account of a case agrees with the whole model and at which scale, and whether it helps machine readers learn the model. Over seven explainers, four datasets, and two model families (mean ± standard deviation over eight settings), the best explainer, KernelSHAP, verified 0.65 ± 0.15 of its claims, while random attributions verified 0.22 ± 0.08. On the same explanations, established faithfulness, remove-and-retrain, and disagreement metrics missed impossible evidence and narratives contradicting their own attributions, and sign-blind metrics scored sign-flipped explanations like the originals. Claim tests withstood scaffold attacks that adapt to the horizon and to the test itself; a scaffold that memorises the audited records weakened them and deceived every explainer, but part–whole agreement still flagged it. HCT is a reproducible screen to apply before user studies.
Language: English
Open Access status: gold
Estimated APC paid: $2,172
OpenAlex ID: W7234297902
FWCI:
Cites: 38
Cited by:
Related to: 10