Pandora: an Ai Model for the Automatic Extraction of Clinical Unstructured Data and Clinical Risk Score Implementation
Year: 2025
Type: conference-paper
Abstract: Introduction Medical records and physician notes contain valuable non-tabular information that requires significant manual effort to extract and structure. Large Language Models (LLMs) have demonstrated the ability to understand, reason, and retrieve information from such sources, transforming it into accessible information for clinical or research purposes. Objective We present PANDORA, an AI system comprising two LLMs that can extract data and use it in validated calculators and prediction models to provide recommendations. Methods The study evaluates the model's ability to extract clinical features from discharge notes from the MIMIC database and synthetically generated outpatient charts. We use the PUMA calculator for Chronic Obstructive Pulmonary Disease (COPD) case finding, which interacts with the model to calculate a score based on seven criteria and determine which patients should undergo further spirometry testing. Results The model’s exhibited excellent extraction accuracy, achieving 100% for MIMIC and 99% for synthetic cases. Interaction with the PUMA scale produced accurate scores, with an accuracy of 94% for both databases. The final recommendation on COPD risk was based on the PUMA scale, classified as positive if score ≥ 5. Sensitivity was 86% for MIMIC and 100% for synthetic cases. Conclusion LLMs have been successful in extracting information and generating recommendations. However, this is the first model that successfully extracts information based on existing, validated clinical scores from plain, non-tabular data and provides a recommendation mixing all these capabilities. It leverages existing knowledge, making it available to be explored in light of the highest-quality evidence in several medical fields.
Authors: Daniel Jimenez, Natalia Castano-Villegas, Isabella Llano, Julian Martinez, Laura Ortiz, Laura Velasquez, Jose Zea
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W4412081980