A Text-To-Text Alignment Algorithm for Better Evaluation of Modern Speech Recognition Systems
Year: 2025
Type: preprint
Abstract: Modern neural networks have greatly improved performance across speech recognition benchmarks. However, gains are often driven by frequent words with limited semantic weight, which can obscure meaningful differences in word error rate, the primary evaluation metric. Errors in rare terms, named entities, and domain-specific vocabulary are more consequential, but remain hidden by aggregate metrics. This highlights the need for finer-grained error analysis, which depends on accurate alignment between reference and model transcripts. However, conventional alignment methods are not designed for such precision. We propose a novel alignment algorithm that couples dynamic programming with beam search scoring. Compared to traditional text alignment methods, our approach provides more accurate alignment of individual errors, enabling reliable error analysis. The algorithm is made available via PyPI.
Source: arXiv (Cornell University)
Authors: Borgholt, Lasse, Havtorn, Jakob, Igel, Christian, Maaløe, Lars, Tan, Zheng-Hua
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Funder: Innovationsfonden
Award: G4822361497
OpenAlex ID: W7084165155
Cites:
Cited by:
Related to: