QLoRA Fine-Tuning and Retrieval-Augmented Generation for Turkish Medical Question Answering: An Evaluation of Corpus Mismatch and Clinical Safety
Year: 2026
Type: conference-paper
Abstract: Large language models (LLMs) have demonstrated remarkable capabilities in clinical knowledge tasks; however, existing systems are predominantly English-centric, leaving a significant gap for low-resource languages such as Turkish. This paper presents a hallucination-aware evaluation framework for Turkish medical question answering (QA), combining Quantized Low-Rank Adaptation (QLoRA)-based parameter-efficient fine-tuning with a Retrieval-Augmented Generation (RAG) pipeline. We fine-tuned Llama-3.1-8B-Instruct on Turkish medical QA data using 4-bit NF4 quantization (v1: 2,675 MedTurkQuAD examples; v2: 13,230 augmentation-expanded examples), and constructed a RAG corpus from 18,577 Turkish hospital articles indexed with multilingual-E5-large embeddings and FAISS. Two model variants (v1 and v2) are evaluated under two modes: Mode A (original context provided) and Mode B (RAG-based retrieval). v1 achieves a semantic similarity of 0.966 (Mode A) and 0.874 (Mode B), alongside a grounding score of 0.715. Under a consistent evaluation methodology (n=50, semantic similarity-based inconsistency), both models exhibit low hallucination rates (v1: 0.051, v2: 0.022), both satisfying the clinical safety threshold of ≤0.20. A systematic corpus mismatch between academic gold answers and clinical web-based RAG sources is identified as the primary driver of low ROUGE-L/Recall in Mode B-a finding supported by Pearson correlation analysis (r=0.940 in Mode A vs. r=0.658 in Mode B) and with broad implications for evaluation methodology in medical AI. These results demonstrate the feasibility of building reliable, evidence-grounded Turkish medical AI assistants using parameterefficient methods on consumer-grade hardware.
Institutions: Yozgat Bozok Üniversitesi, Bakırçay Üniversitesi
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7169881898