Work

A pragmatic risk-stratified framework for using large language models in intensive care medicine: A narrative review

Year: 2026

Type: review

Abstract: Objective To provide Australian intensive care clinicians with a pragmatic framework for the safe integration of large language models (LLMs) into intensive care unit (ICU) practise, addressing the current lack of Australian-specific guidance and limited local evidence. Design Narrative review. Data sources Peer-reviewed publications, preprints, and policy documents relating to LLM use in health care, with a focus on critical care applications and governance. Review methods Evidence and expert commentary were synthesised to develop a clinician-led, risk-stratified framework for ICU implementation, with emphasis on safety, oversight, and applicability within Australian health systems. Clinical use cases, risks, governance considerations, and practical safeguards for day-to-day ICU practise were identified. Results LLMs have potential utility in data-dense ICU environments, including summarising complex clinical information, supporting documentation, assisting clinical reasoning, and facilitating research tasks. However, evidence for LLM performance in ICU contexts remains limited, particularly in Australia. Key risks include inaccurate or fabricated outputs (“ hallucinations ”), bias, lack of situational awareness, privacy concerns, and over-reliance in high-stakes decision-making. We propose a risk-stratified framework that categorises LLM applications by clinical risk and reversibility, and aligns each category with proportional oversight, verification processes, and governance safeguards, emphasising clinician-in-the-loop decision-making. Conclusions LLMs may serve as adjunctive cognitive tools in Australian ICUs when used in clearly defined, low-to intermediate-risk contexts under clinician oversight. Safe integration requires robust governance frameworks emphasising transparency, data protection, and proportionate clinician decision-making. Further Australian-based evaluation is needed before high-risk clinical applications can be considered for routine practise.

Source: Critical Care and Resuscitation

DOI: 10.1016/j.ccrj.2026.100194

Authors: Nilesh Anand Devanand, Santosh Verghese, Stephen Bacchi, Christopher Bain, Ashwin Subramaniam, Krishnaswamy Sundararajan