Get started
Work

Integrating Explainable AI with Retrieval Augmented Generation for Reliable and Personalized Medical Decision Support

Year: 2025
Type: dissertation
Abstract: The following study explores the integration of Large Language Models (LLMs) into the medical domain by combining Explainable AI (XAI) techniques with Retrieval Augmented Generation (RAG) to deliver reliable and personalized responses that account for the user’s cognitive abilities and background knowledge. One of the key components of the system is the MedRAG toolkit, a Retrieval-Augmented Generation (RAG) framework designed for the medical domain, used for improving response quality by integrating information retrieved from medical documents, which has been extended to consider patient clinical notes as contextual input. This allows the model to provide both predictions and detailed explanations that are based on a combination of patient specific information and external medical knowledge retrieved through the RAG mechanism. The system was validated using the MIMIC QA datasets, a subset of the MIMIC III dataset which contains clinical notes and corresponding reference diagnoses validated by human medical experts, and further tested on the PIMA dataset to predict diabetes. Machine learning models such as Random Forest, Decision Tree, CatBoost, and Neural Network were trained, with explanations generated using SHAP and LIME for both local and global interpretability. The system was evaluated using metrics such as BERT Score, BLEU and ROUGE, demonstrating that combining RAG with XAI improves the model’s performance in providing accurate and understandable medical predictions. This study also highlights that traditional XAI methods may not be sufficient in complex domains like healthcare, where the added context from RAG and LLMs offers significant advantages for interpretability and diagnostic support.
Author: PATERNITI BARBINO, NIKO
Language: English
Open Access status: green
OpenAlex ID: W6982633341
Cites:
Cited by:
Related to: