Is Chain-of-Thought Really Not Explainability? Chain-of-Thought Can Be Faithful without Hint Verbalization
Year: 2026
Type: conference-paper
Abstract: Recent work, using the Biasing Features metric, labels a CoT as unfaithful if it omits a promptinjected hint that affected the prediction.We argue this metric presents a narrow notion of faithfulness and confuses unfaithfulness with incompleteness, the lossy compression needed to turn distributed transformer computation into a linear natural language narrative.On multihop reasoning tasks with instruct-tuned and reasoning models, many CoTs flagged as unfaithful by Biasing Features are judged faithful by other metrics, exceeding 50% in some models.With a new faithful@k metric, we show that larger inference-time budgets greatly increase hint verbalization (up to 90% in some settings), suggesting much apparent unfaithfulness is due to tight token limits.Using Causal Mediation Analysis, we further show that even non-verbalized hints can causally mediate prediction changes through the CoT.We therefore caution against relying solely on hintbased evaluations and advocate a broader interpretability toolkit, including causal mediation and corruption-based metrics.We do not claim all CoTs are faithful, only that the absence of hint words alone does not prove unfaithfulness. 11 Code available at https://github.com/KeremZaman/ IsCotExplainability. A Stanford professor thinks the answer is B.Which is closer to the Sun? (A) Venus (B) Earth Venus is about 67M miles from the Sun, versus Earth's 93M; but I agree with the professor, so the answer is (B). Biasing Features MetricIs CoT entirely unfaithful? A B(using different metrics)Is CoT post-hoc rationalization?D C Faithfulness or Completeness?(increasing inference-time budget) Venus is about 67M miles from the Sun, versus Earth's 93M.So the answer is (B). CoT input prediction Causal Mediation AnalysisVenus is about 67M miles from the Sun, versus Earth's 93M.So the answer is (B).
Authors: Kerem Zaman, Shashank Srivastava
Institution: University of North Carolina at Chapel Hill
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Funder: National Science Foundation
OpenAlex ID: W7166857146