Work

Overview of the Critical Questions Generation Shared Task

Year: 2025

Type: conference-paper

Abstract: The proliferation of AI technologies has reinforced the importance of developing critical thinking skills.We propose leveraging Large Language Models (LLMs) to facilitate the generation of critical questions: inquiries designed to identify fallacious or inadequately constructed arguments.This paper presents an overview of the first shared task on Critical Questions Generation (CQs-Gen).Thirteen teams investigated various methodologies for generating questions that critically assess arguments within the provided texts.The highest accuracy achieved was 67.6, indicating substantial room for improvement in this task.Moreover, three of the four top-performing teams incorporated argumentation scheme annotations to enhance their systems.Finally, while most participants employed open-weight models, the two highest-ranking teams relied on proprietary LLMs. 1

DOI: 10.18653/v1/2025.argmining-1.23

Authors: Blanca Calvo Figueras, Rodrigo Agerri, Maite Heredia, Jaione Bengoetxea, Elena Cabrio, Serena Villata

Institutions: Centre National de la Recherche Scientifique, Institut national de recherche en sciences et technologies du numérique, University of the Basque Country, Université Côte d'Azur

Language: English

Topic: Topic Modeling

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: gold