Year: 2018
Type: conference-paper
Abstract: In the last few years Automatic Question Generation (AQG) has attracted increasing interest.In this paper we survey the evaluation methodologies used in AQG.Based on a sample of 37 papers, our research shows that the systems' development has not been accompanied by similar developments in the methodologies used for the systems' evaluation.Indeed, in the papers we examine here, we find a wide variety of both intrinsic and extrinsic evaluation methodologies.Such diverse evaluation practices make it difficult to reliably compare the quality of different generation systems.Our study suggests that, given the rapidly increasing level of research in the area, a common framework is urgently needed to compare the performance of AQG systems and NLG systems more generally.
DOI: 10.18653/v1/w18-6537
Authors: Jacopo Amidei, Paul Piwek, Alistair Willis
Institution: The Open University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W2902583760