Year: 2026
Type: article
Abstract: Medical imaging underpins modern diagnosis, yet the interpretation of scans such as brain MRI, chest radiographs, dermoscopic images, and retinal fundus photographs still depends on scarce specialist expertise, and the computational tools that support it are usually built as isolated, single-modality applications. This paper presents a Multi-Modal AI-Based Healthcare Diagnostic and Clinical Decision Support System that brings four imaging modalities into one explainable and knowledge-grounded platform. Uploaded images are standardised and analysed by a pretrained DenseNet121 backbone; Gradient-weighted Class Activation Mapping (Grad-CAM) converts the network’s internal activations into a spatial heatmap from which condition-wise scores and a leading condition are derived. The leading condition then drives a Retrieval-Augmented Generation (RAG) pipeline, built on a ChromaDB vector store with all-MiniLM-L6-v2 sentence embeddings, that returns curated findings, symptoms, and recommended clinical actions rather than freely generated text. Each analysis is logged in SQLite and can be exported as a structured PDF report through a Streamlit interface. The architecture is modular, traceable at every step, and deliberately positioned as decision support rather than a replacement for clinical judgement. We describe the design, implementation, and functional evaluation of the prototype, discuss its limitations, and outline the path toward clinically validated deployment.
DOI: 10.32628/cseit261269
Authors: Naman Mishra, Abhishek Rai, Yusuf Perwej
Institution: Shri Ramswaroop Memorial University
Language: English
Subfield: Health Informatics
Field: Medicine
Domain: Health Sciences
Open Access status: diamond
OpenAlex ID: W7230943767
