Work

Adding Robust Code-Switching Capabilities to High Performance Multilingual ASR

Year: 2026

Type: preprint

Abstract: Code-switching (CSW) remains challenging for large multi-lingual ASR systems in real-world deployment. While fine-tuning on synthetic CSW data is possible, it generally degrades strong monolingual baselines. Our goal is to preserve these capabilities while extending models to handle complex code-switching, including morphological variations across languages. We propose Bayesian factorized adaptation, which learns to efficiently integrate switching-relevant knowledge into strong pretrained models without overwriting existing capabilities. Requiring only a small amount of synthetic data, our approach reduces transcription errors by 32.87% on code-switched words while improving overall WER by 5.31%, all while maintaining mono-lingual performance. Our results demonstrate that effective CSW adaptation depends more on knowledge integration than data complexity.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.2606.21990

Authors: Ugan, Enes Yavuz, Waibel, Alexander

Institutions: Karlsruhe Institute of Technology, Human Computer Interaction (Switzerland), Carnegie Mellon University

Topic: Speech Recognition and Synthesis

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funders: European Commission, Deutsche Forschungsgemeinschaft, Bundesministerium für Bildung und Forschung

OpenAlex ID: W7165662934

Cites:

Cited by:

Related to: