Work

Task-Specific QLoRA Adaptation in RAG-Based Information Extraction from Research Articles

Year: 2026

Type: article

Abstract: Large Language Models have shown considerable potential for academic document understanding, but their use in information extraction remains limited by hallucination and incomplete evidence grounding. This study presents an end-to-end Retrieval-Augmented Generation framework using a domain-adapted Llama 3.1 8B Instruct model. Generation reliability is evaluated through faithfulness, answer relevancy, context precision, and context recall, while retrieval relevance is measured using a rank-based percentile score that supports comparison across cosine, dot-product, and Euclidean similarity functions. Context recall is computed through a reference-free and answer-dependent procedure and is therefore interpreted as the extent to which the information expressed in an answer is accounted for by the supplied context. Experiments were conducted on 100 research articles using three tasks: research objective extraction, methodological information extraction, and experimental result interpretation. A separate QLoRA adapter was trained for each task, and the base and adapted models were evaluated under identical retrieval and generation conditions across three embedding models and three similarity functions. Under the highest-scoring configuration for each task, the composite score increased from 0.809 to 0.821, from 0.807 to 0.818, and from 0.795 to 0.814, respectively. The adapted model achieved a composite score in 26 of 27 task–retrieval configurations, although only the experimental-result task remained statistically significant after false discovery rate correction. Error analysis showed fewer fabricated statements and less task drift than the base model.

Source: Applied Sciences

DOI: 10.3390/app16168294

Authors: Yılmaz Dikilitaş, Ahmet Sayar, Furkan Göz

Institutions: Kocaeli Üniversitesi, Bahçeşehir University

Language: English

Topic: Topic Modeling

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: gold

Estimated APC paid: $2,896

Funder: Kocaeli Üniversitesi