Overview of ToxHabits at BioCreative IX: corpus, guidelines and evaluation of systems for the detection of Toxic Habits from text
Year: 2025
Type: conference-paper
Abstract: Systems able to detect mentions of substance use and abuse from clinical texts are essential for healthcare analytics, substance use surveillance, and AI-based decision support. Compared to other clinical concepts, substance use and behavioral information are often only captured in free-text narratives and are rarely structured in electronic health records. Despite their relevance for clinical research and public health, few annotated corpora exist for substance use detection, and prior efforts have been limited by narrow domain coverage, English-only datasets, or lack of event-based annotations. To address these limitations, we proposed the ToxHabits shared task. The task is based on a Spanish corpus of 1,499 clinical case reports covering multiple medical specialties, with event-based annotations for substance use and associated attributes. The task is divided into two subtasks: ToxNER, focused on identifying substance use mentions (triggers), and ToxUse, aimed at extracting related information that characterizes the substance use (arguments), such as type, frequency, and route. A total of 6 teams participated, each allowed to submit up to 5 runs per subtask. The top-scoring systems achieved an F1-score of 0.94 for ToxNER and 0.91 for ToxUse. Considering the complexity of real-world narratives, nested entities, and fine-grained event structures, these results demonstrate the viability of accurate substance use detection from clinical text and offer a strong benchmark for future work in Spanish-language biomedical NLP. The ToxHabits Gold Standard is freely available at: https://zenodo.org/records/15425460. This article is part of the Proceedings of the BioCreative IX Challenge and Workshop (BC9): Large Language Models for Clinical and Biomedical NLP at the International Joint Conference on Artificial Intelligence (IJCAI).
Authors: Al-Nabki, Mhd Wesam, Lima López, Salvador, Vayá-Abad, Gabriel, Krallinger, Martin
Institution: Barcelona Supercomputing Center
Language: English
Subfield: Molecular Biology
Domain: Life Sciences
Open Access status: green
OpenAlex ID: W7077076227