TICL: Text-Embedding KNN for Speech in-Context Learning Unlocks Speech Recognition Abilities of Large Multimodal Models
Year: 2026
Type: conference-paper
Abstract: Speech foundation models have recently demonstrated the ability to perform Speech In-Context Learning (SICL). Selecting effective in-context examples is crucial for SICL performance, yet selection methodologies remain underexplored. In this work, we propose Text-Embedding KNN for SICL (TICL), a simple pipeline that uses semantic context to enhance off-the-shelf large multimodal models’ speech recognition abilities without fine-tuning. Across challenging automatic speech recognition tasks, including accented English, multilingual speech, and children’s speech, our method enables models to surpass zero-shot performance up to 84.7% relative WER reduction. Ablation studies are conducted to show the robustness and efficiency of our method.
Authors: Haolong Zheng, Yekaterina Yegorova, Mark Hasegawa-Johnson
Institution: University of Illinois Urbana-Champaign
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Funder: National Science Foundation
OpenAlex ID: W4416259209