NoviceTrio in #SMM4H-HeaRD 2026: Hybrid Clinical Transformer Ensembles for Insomnia Detection and Evidence Extraction from Clinical Notes
Year: 2026
Type: conference-paper
Abstract: We present two systems for the #SMM4H-HeaRD 2026 Task 2 (Lopez-Garcia et al., 2026) shared task of automated insomnia detection from clinical notes.Our system addresses both subtasks: (1) binary insomnia classification and (2) multi-label rule prediction with evidence span extraction.For Subtask 1, we employ an ensemble architecture combining Qwen3-4B-Instruct 1 and Bio_ClinicalBERT 2 to capture both general semantic reasoning and domainspecific clinical representations.The framework utilizes chunk-based processing with overlapping token windows to handle long clinical notes efficiently.For Subtask 2, we develop a dual-head multi-task transformer model that jointly predicts insomnia labels and token-level evidence spans using BIO tagging 3 .To improve clinical relevance, we additionally incorporate sentence-level filtering using sentencetransformer embeddings and similarity-based retrieval of insomnia-related contexts.Experimental results suggest competitive performance relative to the shared task mean and median scores across both subtasks.Our best Subtask 1 system achieves a recall of 0.9474, surpassing the shared-task mean and median recall, while our Subtask 2 system exceeds the mean and median scores across label classification, exact match, and partial match metrics.The endto-end implementation is publicly available on GitHub 4 .
Authors: Abir Naskar, Mike Conway
Institution: The University of Melbourne
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166830179