Work

Execution of Harm-Enabling Actions by LLM Agents During Simulated Psychiatric Crises

Year: 2026

Type: preprint

Abstract: Agent-enabled large language models (LLMs) can autonomously execute user-directed actions, raising new safety concerns in mental health contexts. Using a structured algorithmic safety audit with clinically validated vignettes, we examined whether an agent-enabled LLM executes harm-enabling actions during simulated psychiatric crises. The model executed such actions in over half of the trials (54%). These findings highlight execution-level risks that extend beyond conversational safety and warrant explicit safeguards for agentic systems in mental health–relevant settings.

Source: Research Square

DOI: 10.21203/rs.3.rs-8743363/v1

Authors: Ziv Ben-Zion, David Piterman, Elad Refoua, Zohar Elyoseph

Institutions: University of Haifa, Bar-Ilan University

Language: English

Topic: Ethics and Social Impacts of AI

Subfield: Safety Research

Field: Social Sciences

Domain: Social Sciences

SDG: Good health and well-being

Open Access status: green

OpenAlex ID: W7127045028

Cites: 15

Cited by: 1

Related to: 10