Large language models for hantavirus-related infectious disease risk communication during the 2026 Andes virus outbreak: an expert-rated comparative content analysis
Year: 2026
Type: article
Abstract: Large language models (LLMs) are increasingly used for health information, but their reliability in outbreak-related public health communication remains uncertain. Hantavirus communication is challenging because messages must balance avoidance of unnecessary alarm with prevention of false reassurance, particularly during the 2026 Andes virus outbreak. This comparative, cross-sectional, expert-rated content analysis evaluated responses generated by three non-Pro publicly accessible LLMs - ChatGPT, Gemini and Claude - to 150 predefined hantavirus-related questions. Questions covered official health authority/guideline-based information, expert/clinical practice issues and public frequently asked questions. Each question was submitted using a standard zero-shot prompt and a public health risk communication prompt, generating 900 responses. Three independent experts assessed accuracy, completeness, consistency, safety and actionability using a predefined rubric. Ratings were summarised descriptively, and inter-rater reliability was assessed using percentage agreement and Fleiss’ kappa. Overall, 568 of 900 responses (63.1%) were rated as completely correct and sufficient, 232 (25.8%) as mostly correct but incomplete, and 100 (11.1%) as partially correct or potentially misleading. No response was rated as completely incorrect or overtly harmful. Within this dataset, ChatGPT had numerically the highest proportion of completely correct responses (198/300, 66.0%) and the lowest proportion of misleading responses (28/300, 9.3%); however, these differences were interpreted descriptively rather than as evidence of model superiority. Gemini and Claude had completely correct response rates of 62.7% and 60.7%, respectively. In this dataset, the generic public health risk communication prompt did not change the aggregate expert-rated categorical distribution. Major publicly accessible LLMs generated a high proportion of correct or mostly correct hantavirus risk communication responses, but approximately one in nine responses was potentially misleading. These tools may support infectious disease and outbreak-related risk communication but should not be used as stand-alone sources for clinically complex or rapidly evolving guidance without expert oversight and official-source grounding. Not applicable.
Source: BMC Infectious Diseases
Language: English
Subfield: Health Informatics
Field: Medicine
Domain: Health Sciences
Open Access status: gold
Estimated APC paid: $3,090
OpenAlex ID: W7212280925
FWCI:
Cites:
Cited by:
Related to: