Year: 2025
Type: conference-paper
Abstract: Recent advances in speech foundation models are largely driven by scaling both model size and data, enabling them to perform a wide range of tasks, including speech recognition.Traditionally, ASR models are evaluated using metrics like Word Error Rate (WER) and Character Error Rate (CER), which depend on ground truth labels.As a result of limited labeled data from diverse domains and testing conditions, the true generalization capabilities of these models beyond standard benchmarks remain unclear.Moreover, labeling data is both costly and time-consuming.To address this, we propose a novel label-free approach for approximating ASR performance metrics, eliminating the need for ground truth labels.Our method utilizes multimodal embeddings in a unified space for speech and transcription representations, combined with a high-quality proxy model to compute proxy metrics.These features are used to train a regression model to predict key ASR metrics like Word Error Rate (WER) and Character Error Rate (CER).We experiment with over 40 models across 14 datasets representing both standard and in-thewild testing conditions.Our results show that we approximate the metrics within a singledigit absolute difference across all experimental configurations, outperforming the most recent baseline by more than 50%.MD Rio, Peter Ha, Quinten McNamara, Corey Miller, and Shipra Chandra.2022.Earnings-22: A practical benchmark for accents in the wild.
Authors: Abdul Waheed, Hanin Atwany, Rita Singh, Bhiksha Raj
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
OpenAlex ID: W4412944683