Open ASR Leaderboard: Towards Reproducible and Transparent Multilingual and Long-Form Speech Recognition Evaluation
Year: 2026
Type: conference-paper
Abstract: We present the Open ASR Leaderboard, a reproducible benchmarking platform with community contributions from academia and industry.It compares 85+ open-source and proprietary systems across 11 datasets, with English short-and long-form and multilingual short-form tracks.We standardize word error rate (WER) and inverse real-time factor (RTFx) evaluation for consistent accuracy-efficiency comparisons across model architectures and toolkits (e.g., ESPNet, NeMo, SpeechBrain, Transformers).We observe that Conformer-based encoders paired with large language model (LLM)-based decoders achieve the best average WER, while connectionist temporal classification (CTC) and token-and-duration transducer (TDT) decoders offer superior RTFx, making them better suited for long-form and batched processing.All code and dataset loaders are open-sourced to support transparent, extensible evaluation.We present our evaluation methodology to facilitate communitydriven benchmarking in ASR and other tasks.
Source: Interspeech (USB)
Authors: Vaibhav Srivastav, Steven Zheng, Eric Bezzam, Eustache Le Bihan, Nithin Rao Koluguri, Piotr Żelasko, Somshubra Majumdar, Adel Moumen, Sanchit Gandhi
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W4415097571