Work

In-Context Meta-Learning vs. Semantic Score-Based Similarity: A Comparative Study in Arabic Short Answer Grading

Year: 2023

Type: conference-paper

Abstract: Delegating short answer grading to automated systems enhances efficiency, giving teachers more time for vital human-centered aspects of education.Studies in automatic short answer grading (ASAG) approach the problem from instance-based or reference-based perspectives.Recent studies have favored instance-based methods, but they demand substantial data for training, which is often scarce in classroom settings.This study compares both approaches using an Arabic ASAG dataset.We employ incontext meta-learning for instance-based and semantic score-based similarity for referencebased grading.Results show both methods outperform a baseline and occasionally even surpass human raters when grading unseen answers.Notably, the semantic score-based similarity approach excels in zero-shot settings, outperforming in-context meta-learning.Our work contributes insights to Arabic ASAG and introduces a prompt category classification model, leveraging GPT3.5 to augment Arabic data for improved performance.

Source: Arabic Natural Language Processing Conference (ArabicNLP)

DOI: 10.18653/v1/2023.arabicnlp-1.28

Authors: Menna Fateen, Tsunenori Mina

Institution: Kyushu University

Language: English

Topic: Educational Technology and Assessment

Subfield: Information Systems

Field: Computer Science

Domain: Physical Sciences

SDG: Quality education

Open Access status: hybrid

Funder: Japan Society for the Promotion of Science