Work

Calibrated reliance on generative-AI health advice: a perspective on correctness, risk, and escalation

Year: 2026
Type: article
Abstract: Members of the public increasingly consult generative artificial intelligence (AI) for health advice, and research has largely asked whether they trust it. Trust is the wrong yardstick. Reliance on health advice is appropriate only when it tracks whether the advice is correct and what an error would cost. We argue for an outcome standard, calibrated reliance, under which four actions (adopt, verify, reject, and escalate to professional care) are judged jointly against advice correctness and the cost of error. Drawing on trust calibration, automation bias, algorithm aversion, advice taking, and heuristic models of credibility, we read a purposive set of human-participant studies against this standard. The studies show both reliance errors. Identical advice is discounted when labeled as AI-generated, which is consistent with underreliance. Low-accuracy AI responses are rated and followed much like physicians' responses, and lay users assisted by large language models choose care dispositions no better than controls, which is consistent with overreliance. Both patterns suggest that lay reliance responds to source and fluency cues more than to correctness. Interventions that require users to act, such as verification affordances, cognitive forcing, and expressed uncertainty, show more promise than passive labels, but no study has yet manipulated correctness and graded clinical risk together while observing verification and escalation. We state five propositions with their evidential status, operational definitions, and a factorial design that would test them.
Language: English
Field: Medicine
Open Access status: gold
Estimated APC paid: $3,801
OpenAlex ID: W7235923802
FWCI:
Cites: 37
Cited by:
Related to: 10