Accuracy, completeness, safety, and readability of large language model responses to medication queries in pregnancy and lactation: a cross-sectional comparative evaluation
Year: 2026
Type: preprint
Abstract: This cross-sectional study evaluates the accuracy, completeness, safety, and readability of medicines information generated by four large language models (ChatGPT, Gemini, Claude, DeepSeek) in response to 40 standardised medication queries concerning pregnancy and lactation. Responses are assessed against pre-specified reference answers derived from LactMed and UKTIS/BUMPS by two pharmacists rating independently and blinded to model identity. The primary outcome is the proportion of responses containing no major factual error and no potentially harmful recommendation. The query set, reference answers, and scoring manual are frozen before data collection.
Source: OSF Preprints (OSF Preprints)
Author: Hira Khalid
Language: English
Subfield: Health Informatics
Field: Medicine
Domain: Health Sciences
Open Access status: green
OpenAlex ID: W7172084430