Work

GEESE: Generating and Evaluating Explanations for Semantic Entailment: A CALAMITA Challenge

Year: 2024

Type: conference-paper

Abstract: In the GEESE challenge, we present a pipeline to evaluate generated explanations for the task of Recognizing Textual Entailment (RTE) in Italian. The challenge focuses on evaluating the impact of generated explanations on the predictive performance of language models. Using a dataset enriched with human-written explanations, we employ two large language models (LLMs) to generate and utilize explanations for semantic relationships between sentence pairs. Our methodology assesses the quality of generated explanations by measuring changes in prediction accuracy when explanations are provided. Through reproducible experimentation, we establish benchmarks against various baseline approaches, demonstrating the potential of explanation injection to enhance model interpretability and performance.

Source: Open Research (University of Surrey)

Authors: Zaninello, Andrea, Magnini, B

Institutions: Free University of Bozen-Bolzano, Fondazione Bruno Kessler

Language: English

Topic: Explainable Artificial Intelligence (XAI)

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funders: Agence Nationale de la Recherche, CHIST-ERA

Award: ArgumeNtaTIon-Driven explainable artificial intelligence fOr digiTal mEdicine

OpenAlex ID: W7133269027