Performance evaluation of Meditron3-70B in medical coding: Current limitations and integration perspectives for clinical practice
Year: 2025
Type: article
Abstract: The emergence of large language models (LLMs) presents challenges for their integration into clinical practice, particularly for medical coding. This study evaluated the performance of Meditron3-70B, a state-of-the-art open-source medical LLM, in generating SNOMED CT and ICD-10 codes using 200 fictitious emergency department consultation vignettes. Healthcare professionals assessed the accuracy of the outputs. Although Meditron performed well on standard benchmarks such as MedQA, significant shortcomings were observed in the relevance and completeness of the generated diagnostic codes, with only 2% of responses deemed acceptable. While LLMs show promise for supporting clinical decision-making, their current ability to produce accurate and comprehensive medical codes remains limited. Integrating specialized retrieval tools through hybrid approaches could enhance coding accuracy and warrants further investigation in real-world clinical settings.
DOI: 10.3205/mibe000287
Authors: Galland-Decker, Coralie, Usenbacher, Muaziza, Nunes, Christophe, Bouche, François, Carra, Giorgia, Boillat-Blanco, Noémie, Hartley, Mary-Anne, Sallinen, Alexandre, Raisaro, Jean Louis, Bastardot, François
Language: English
Subfield: Health Informatics
Field: Medicine
Domain: Health Sciences
Open Access status: green
OpenAlex ID: W7092293490
FWCI:
Cites:
Cited by:
Related to: