A Lightweight Audio-LLM Framework for Infant Cry Behavior Recognition Using Wav2Vec2 and TinyLlama
Year: 2026
Type: preprint
Abstract: Infant crying is a primary form of non-verbal communication through which infants express physiological and behavioral needs. However, accurately interpreting infant cries remains challenging because cry signals exhibit substantial acoustic and temporal variability, particularly under noisy conditions. To address this challenge, we propose a lightweight Audio-Large Language Model (Audio-LLM) framework for infant cry behavior recognition by integrating a pre-trained Wav2Vec2-base audio encoder with the TinyLlama-1.1B language model. Wav2Vec2-base extracts high-level acoustic representations directly from the infant cry waveform, which are temporally aggregated and transformed through a linear projection and LayerNorm into an LLM-compatible audio token. The resulting audio representation is combined with a task-specific text prompt and processed by TinyLlama, while Low-Rank Adaptation (LoRA) enables parameter-efficient adaptation of the language model. The framework performs five-class infant cry classification, including belly pain, burping, discomfort, hunger, and tiredness, using the Donate-a-Cry Corpus dataset. The model achieved 99.34\% validation accuracy under the clean training condition. To further evaluate robustness, controlled noise experiments were conducted at 0.1\%, 0.2\%, 0.5\%, and 2\% noise levels, achieving validation accuracies of 95.33\%, 86.00\%, 71.92\%, and 67.42\%, respectively. These results demonstrate the effectiveness of pre-trained acoustic representations combined with a compact LLM for infant cry behavior recognition and provide a quantitative assessment of performance degradation under increasing noise conditions. The proposed framework offers a lightweight alternative to conventional approaches and establishes an Audio-LLM-based direction for infant cry analysis.
Source: Research Square
Institutions: Kookmin University, Islamia College University, Ajou University
Language: English
Field: Medicine
Domain: Health Sciences
Open Access status: green
OpenAlex ID: W7236595575
