Work

How Interpretable are Reasoning Explanations from Prompting Large Language Models?

Year: 2024
Type: conference-paper
Abstract: Prompt Engineering has garnered significant attention for enhancing the performance of large language models across a multitude of tasks.Techniques such as the Chain-of-Thought not only bolster task performance but also delineate a clear trajectory of reasoning steps, offering a tangible form of explanation for the audience.Prior works on interpretability assess the reasoning chains yielded by Chain-of-Thought solely along a singular axis, namely faithfulness.We present a comprehensive and multifaceted evaluation of interpretability, examining not only faithfulness but also robustness and utility across multiple commonsense reasoning benchmarks.Likewise, our investigation is not confined to a single prompting technique; it expansively covers a multitude of prevalent prompting techniques employed in large language models, thereby ensuring a wide-ranging and exhaustive evaluation.In addition, we introduce a simple interpretability alignment technique, termed Self-Entailment-Alignment Chain-of-thought, that yields more than 70% improvements across multiple dimensions of interpretability.Code is available at https://github.com/SenticNet/ CoT_interpretability
Language: English
Open Access status: hybrid
OpenAlex ID: W4401043361
FWCI: 7.928
Cites: 44
Cited by: 102
Related to: 10