Work

Evaluation of the consistency of a speech verification system with human raters in early literacy screening assessments

Year: 2026

Type: article

Abstract: This study investigates the use of a speech verification system (SVS) technology, a form of automatic speech recognition (ASR), in the assessment of children's reading skills. Despite the growing integration of ASR systems in educational assessment, significant challenges persist, particularly due to the acoustic, pronunciation and dialectal variability inherent in children's speech. Our research evaluated the consistency between human rater (HR) and speech verification system (SVS) scores produced from SoapBox Labs across three linguistic tasks—phoneme blending, expressive vocabulary and word reading. Results reveal variability in agreement rates, with SVS showing lower consistency with human raters in phonologically complex tasks like phoneme blending, compared to expressive vocabulary and word reading tasks. Additionally, we address potential racial differences in SVS performance, highlighting the importance of diverse speech sample collection to ensure equitable assessments, as well as inter-item differences within a task. The study concludes with recommendations for consideration of using SVS in educational assessments, advocating for ongoing research and algorithmic advancements to better support educational assessment practices.

Source: Frontiers in Education

DOI: 10.3389/feduc.2026.1671946

Authors: Yaacov Petscher, Jennifer O'Sullivan, Hugh W. Catts, Ashley Edwards, Lisa Fitton

Institutions: Florida State University, University of South Carolina

Language: English

Topic: Speech Recognition and Synthesis

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

SDGs: Quality education, Reduced inequalities

Open Access status: gold

Estimated APC paid: $2,353