Ground Truth Heterogeneity: Exploring Development and Non-Clinical Use Of Retrieval-Augmented Generation Chatbot In Healthcare
Year: 2026
Type: article
Abstract: Retrieval-augmented generation (RAG) architecture enhances large language models (LLMs) by incorporating external, domain-specific data sources, thereby reducing hallucinations and improving response accuracy. This approach departs from purely generative systems and offers a scalable, cost-efficient alternative for enhancing LLMs. However, the construction of ground truth to enable evolvability (ability to integrate new data continuously) and adaptability (ability to adjust responses to user context and intents dynamically) remains challenging. This study investigates how the challenges of establishing ground truth manifest at the organizational, unit, team, and individual levels within a countywide Swedish healthcare organization. Our analysis reveals that the main challenges stem from the heterogeneity of knowledge practices, which complicate the creation of reliable retrieval from the knowledge base. Furthermore, we identify a trade-off between attempts to enhance accuracy and maintain contextual relevance. These findings contribute practical insights into the micro-foundations of evolvability and adaptability in RAG-based chatbot development.
Source: AIS Electronic Library (AISeL)
Institution: Umeå University
Language: English
Subfield: Health Informatics
Field: Medicine
Domain: Health Sciences
Open Access status: green
OpenAlex ID: W7162610678
