Work

Safety validation of large language models for mental health crisis intervention in oncology and general populations: a stratified active-learning study (Preprint)

Year: 2025
Type: preprint
Abstract: BACKGROUND Large language models are increasingly deployed in mental health crisis intervention, where safety validation requires expert evaluation across diverse populations. Oncology patients face elevated crisis risk due to diagnosis trauma and existential distress, yet active learning approaches prioritizing high-uncertainty cases may systematically exclude specific populations from validation. OBJECTIVE To evaluate whether stratified active learning eliminates systematic validation gaps across oncology and general populations in mental health crisis intervention systems while maintaining practical efficiency. METHODS We evaluated an ensemble-based language model across 720 clinical vignettes (360 oncology, 360 general population) with three board-certified clinicians and psychiatrist along with independent validation by two US-based physicians. Vignettes spanned six crisis types (abuse disclosure, imminent suicide, self-harm, substance intoxication, non-suicidal self-injury, psychosis) at three severity levels, creating 36 population-by-domain-by-severity combinations. We compared stratified active learning against pure uncertainty sampling for validation coverage. RESULTS Pure uncertainty sampling achieved only 44% coverage, completely excluding oncology patients with psychosis and non-suicidal self-injury despite these representing 35% of oncology crisis cases with equal model uncertainty to included combinations (mean 0.701 vs 0.759, P=0.042). Stratified active learning achieved 100% coverage (χ²=15.8, P<0.001) while reducing annotation burden by 92% (12 vs 144 hours) and maintaining 94% efficiency. CONCLUSIONS Stratified active learning ensures equitable validation across oncology and general populations within practical resource constraints, eliminating systematic exclusion of vulnerable subgroups. Oncology mental health services can deploy validated crisis intervention systems covering all patient presentations rather than accepting gaps that expose cancer patients to unvalidated AI systems.
Language: English
Field: Medicine
Open Access status: green
OpenAlex ID: W4417121237
Cites: 26
Cited by:
Related to: 10