A Century After Jacobson v. Massachusetts: Public Health Ethical Perspectives of Large Language Models on Vaccination
Year: 2025
Type: conference-paper
Abstract: ObjectivesThis study evaluates and compares the ethical reasoning of open-source and closed-source large language models (LLMs) in vaccination ethics, using Jacobson v. Massachusetts as a reference. It assesses the accuracy, soundness, and alignment of LLM responses with those of a human public health expert (PHE), exploring implications for public health decisions. MethodsEthical questions based on Jacobson v. Massachusetts were posed to three LLMs—ChatGPT 4.0, LLaMA 3 (70B), and Gemini 1.5—and a PHE. The LLMs’ yes/no responses, justifications, and accuracy in identifying relevant cases (post-2000) were evaluated. Advanced prompt engineering was employed, with statistical analyses (Cohen’s Kappa, Kruskal-Wallis H test) measuring agreement and significance. ResultsChatGPT and LLaMA showed high agreement with the PHE (Kappa = 0.78), while Gemini had low agreement (Kappa = 0.05). ChatGPT provided 100% accurate case relevance, outperforming LLaMA and Gemini. ConclusionLLMs can yield optimal responses in vaccination ethics with advanced prompt engineering. However, model selection is crucial as performance varies. Persistent “hallucinations” in LLM outputs highlight the need for human validation to ensure accurate public health decision-making.
Language: English
Subfield: Health Informatics
Field: Medicine
Domain: Health Sciences
Open Access status: green
OpenAlex ID: W4412934175