Year: 2026
Type: conference-paper
Abstract: Word Error Rate (WER) is the dominant ASR metric but can overestimate errors when references and hypotheses encode the same words in different scripts, a common issue in multilingual settings where models emit romanized text.We propose Script-Normalized WER (SN-WER), a training-free, evaluation-only scoring method that transliterates reference and hypothesis into a language-specific canonical script before computing WER.Evaluated on 5 Indic languages, 2 datasets, and 3 ASR models, SN-WER quantifies script mismatch effects: on curated FLEURS it narrows inflated model gaps by up to 12%, while on noisy Common Voice the reductions are smaller or inconsistent, exposing genuine recognition weaknesses rather than only script mismatch.Controlled stress tests show 67% attenuation of artificial romanization-induced WER inflation, while lexical-substitution controls show near-identical sensitivity to semantic errors (∆SN/∆WER ≈ 1.09).SN-WER is robust to transliterator choice (∆ < 0.002), normalization changes (∆ < 0.05), and has low tokencollision rates (< 0.1%) in the evaluated Indic setting.We argue that SN-WER should be reported alongside WER/CER as a companion score for script-insensitive ASR evaluation, especially when transcripts feed downstream search, indexing, or multilingual LLM pipelines.
Author: Priyaranjan Pattnayak
Institution: Oracle (United States)
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166806359