Year: 2022
Type: preprint
Abstract: Large language models have achieved high performance on various question answering (QA) benchmarks, but the explainability of their output remains elusive. Structured explanations, called entailment trees, were recently suggested as a way to explain and inspect a QA system's answer. In order to better generate such entailment trees, we propose an architecture called Iterative Retrieval-Generation Reasoner (IRGR). Our model is able to explain a given hypothesis by systematically generating a step-by-step explanation from textual premises. The IRGR model iteratively searches for suitable premises, constructing a single entailment step at a time. Contrary to previous approaches, our method combines generation steps and retrieval of premises, allowing the model to leverage intermediate conclusions, and mitigating the input size limit of baseline encoder-decoder models. We conduct experiments using the EntailmentBank dataset, where we outperform existing benchmarks on premise retrieval and entailment tree generation, with around 300% gain in overall correctness.
Source: arXiv (Cornell University)
Authors: Ribeiro, Danilo, Wang, Shen, Ma, Xiaofei, Dong, Rui, Wei, Xiaokai, Zhu, Henry, Chen, Xinchi, Huang, Zhiheng, Xu, Peng, Arnold, Andrew, Roth, Dan
Institutions: Northwestern University, Amazon (United States)
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Funder: Office of Naval Research
OpenAlex ID: W4281250596