Year: 2026
Type: conference-paper
Abstract: Current end-to-end ASR models excel at general transcription but often struggle with context-specific keywords. Existing contextual ASR methods are typically categorized into token-level and phrase-level approaches. Token-level methods generate text step-by-step to provide fine-grained control for generating both keywords and general tokens, but often struggle to preserve the integrity of keyword phrases. In contrast, phrase-level methods model keywords as indivisible units to preserve phrase integrity, yet often degrade the recognition of non-keyword text. Prior work has not explored combining these complementary strategies. To address this gap, we propose an end-to-end multi-grained fusion framework that systematically integrates a large language model (LLM) at both granularities. For token-level fusion, it fuses the token probabilities of the ASR model and the LLM, while for phrase-level fusion, it jointly leverages the ASR model’s acoustic evidence and the LLM’s semantic guidance to select keyword phrases. A joint multi-grained mechanism then chooses the optimal output at each decoding step. Experiments on Chinese and English datasets show our approach achieves up to a 1.2% absolute error reduction and 9.8% keyword recall improvement. Ablation studies confirm both token-level and phrase-level components are essential and complementary to the final performance.
Authors: Shilin Zhou, Zhenghua Li
Institution: Soochow University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W4415142272
FWCI:
Cites:
Cited by:
Related to: