BanglaAudioNative: Toward an End-to-End, Text-Free, Full-Duplex Speech Model for Bangla
Year: 2026
Type: preprint
Abstract: Traditional conversational AI systems rely on a cascaded pipeline of automatic speech recognition (ASR), a text-based large language model (LLM), and text-to-speech (TTS) synthesis. Although functional, this text-mediated design adds latency at every stage, discards acoustic and emotional cues such as prosody and tone, and handles natural turn-taking poorly, including interruptions and concurrent backchanneling. We propose BanglaAudioNative, a research plan to build an end-to-end, audio-to-audio speech architecture for Bangla that requires no intermediate text transcription at inference. The system would model dual-stream conversational audio through a neural audio codec and an auto-regressive multi-stream transformer, mapping Bangla phonetics and prosody directly from speech to speech. To support training, we plan to build a dual-channel conversational speech dataset of spontaneous, multi-dialect Bangladeshi speech, targeting approximately 50 hours, that includes natural overlaps and Bangla–English (“Benglish”) code-switching. We will adapt an open-source full-duplex speech framework using parameter-efficient fine-tuning, with target objectives of sub-second response latency and real-time interruption handling. If successful, this work would address a gap in end-to-end voice intelligence for Bangla and provide an open blueprint for South Asian full-duplex speech systems.
Author: Mifta, Jubair Rahman
Institution: Stamford University Bangladesh
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225583776