Year: 2025
Type: conference-paper
Abstract: Traditionally, automatic speech recognition (ASR) systems rely on human transcriptions to calculate word error rate (WER) by comparing ASR outputs to manual transcriptions. Recently, Amazon's mobile voice shopping platform stopped storing audio from incoming requests to enhance customer privacy, making offline, human-based evaluation unfeasible. This presentation introduces a multitask Speech LLM-based system that processes real-time audio, extracting key features to track ASR performance and detect traffic shifts-all without storing audio or requiring human annotations. Additionally, we demonstrate how combining these features with a synthetic audio generation model (TTS) enables accurate detection of ASR performance degradation, ensuring continuous optimization of the customer voice experience.
Authors: Dhruv Agarwal, Nupur Neti, Federica Cerina
Institution: Amazon (United States)
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
OpenAlex ID: W4407953196