Work

Consonant-Vowel Transition Models Based on Deep Learning for Objective Evaluation of Articulation

Year: 2022

Type: article

Abstract: Spectro-temporal dynamics of consonant-vowel (CV) transition regions are considered to provide robust cues related to articulation. In this work, we propose an objective measure of precise articulation, dubbed the objective articulation measure (OAM), by analyzing the CV transitions segmented around vowel onsets. The OAM is derived based on the posteriors of a convolutional neural network pre-trained to classify between different consonants using CV regions as input. We demonstrate that the OAM is correlated with perceptual measures in a variety of contexts including (a) adult dysarthric speech, (b) the speech of children with cleft lip/palate, and (c) a database of accented English speech from native Mandarin and Spanish speakers.

Source: IEEE/ACM Transactions on Audio Speech and Language Processing

DOI: 10.1109/taslp.2022.3209937

Authors: Vikram C. Mathad, Julie M. Liss, Kathy Chapman, Nancy Scherer, Visar Berisha

Institutions: Arizona State University, University of Utah

Language: English

Topic: Speech Recognition and Synthesis

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funders: National Institutes of Health, National Institute on Deafness and Other Communication Disorders, National Institute of Dental and Craniofacial Research, National Center for Advancing Translational Sciences

Awards: An inter-center comparison of speech outcomes: The Americleft project, Objectively Quantifying Speech Outcomes of Children with Cleft Palate, Perception of dysarthric speech: An objective model of dysarthric speech evaluation with actionable outcomes, CTSA UM1 Program at University of Utah