Work

Accuracy of online symptom assessment applications, large language models, and laypeople for self–triage decisions

Year: 2025

Type: article

Abstract: Symptom-Assessment Application (SAAs, e.g., NHS 111 online) that assist laypeople in deciding if and where to seek care (self-triage) are gaining popularity and Large Language Models (LLMs) are increasingly used too. However, there is no evidence synthesis on the accuracy of LLMs, and no review has contextualized the accuracy of SAAs and LLMs. This systematic review evaluates the self-triage accuracy of both SAAs and LLMs and compares them to the accuracy of laypeople. A total of 1549 studies were screened and 19 included. The self-triage accuracy of SAAs was moderate but highly variable (11.5-90.0%), while the accuracy of LLMs (57.8-76.0%) and laypeople (47.3-62.4%) was moderate with low variability. Based on the available evidence, the use of SAAs or LLMs should neither be universally recommended nor discouraged; rather, we suggest that their utility should be assessed based on the specific use case and user group under consideration.

Source: npj Digital Medicine

DOI: 10.1038/s41746-025-01566-6

Authors: Marvin Kopka, Niklas von Kalckreuth, Markus A. Feufel

Institution: Technische Universität Berlin

Language: English

Topic: Artificial Intelligence in Healthcare and Education

Subfield: Health Informatics

Field: Medicine

Domain: Health Sciences

SDG: Good health and well-being

Open Access status: gold

Estimated APC paid: $4,090