Work

Retrieval-augmented generation for personalized physician recommendations in online medical services: model development study

Year: 2025

Type: article

Abstract: Web-based medical services have expanded access to healthcare through remote consultations and streamlined scheduling, but personalized physician recommendations remain limited due to reliance on manual triage. This study developed and validated a Retrieval-Augmented Generation-Based Physician Recommendation (RAGPR) model to enhance triage performance. Using 646,383 consultation records from the Internet Hospital of the First Affiliated Hospital of Xiamen University, we evaluated embedding models (FastText, SBERT, OpenAI) for clustering and classification, as well as large language models (Mistral, GPT-4o-mini, GPT-4o). Three triage staff also assessed model efficiency via questionnaires. Results showed that FastText performed poorly (F1-score 46%), while SBERT and OpenAI achieved 95% and 96%. Among LLMs, GPT-4o reached the highest F1-score (95%) with a performance rating of 4.67, followed by Mistral (94%, 4.56) and GPT-4o-mini (92%, 4.45). Considering accuracy, cost, and implementation, SBERT and Mistral were optimal. The RAGPR model offers a scalable approach to improving accuracy and personalization in online patient– physician matching.

Source: Optimizations in Applied Machine Learning

DOI: 10.71070/oaml.v5i1.141

Authors: Yingbin Zheng, Yiwei Yan, Sai Chen, Yunping Cai, Kun Ren, Yishan Liu, Jiaying Zhuang, Min Zhao

Institutions: Xiamen University, First Affiliated Hospital of Xiamen University, China Meteorological Administration, Taiyuan University of Technology

Language: English

Topic: Machine Learning in Healthcare

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

SDG: Good health and well-being

Open Access status: hybrid

OpenAlex ID: W7083593784

FWCI:

Cites:

Cited by:

Related to: