Year: 2026
Type: article
Study design: Systematic Review
Abstract: A ‘silent trial’ refers to the prospective, noninterventional testing of artificial intelligence (AI) models in the intended clinical setting without affecting patient care or institutional operations. The silent evaluation phase has received less attention than in silico algorithm development or formal clinical evaluations, despite its increasing recognition as a critical phase. There are no formal guidelines for performing silent AI evaluations in healthcare settings. We conducted a scoping review to identify silent AI evaluations described in the literature and to summarize current practices for performing silent testing. We screened the PubMed, Web of Science and Scopus databases for articles fitting our criteria for silent AI evaluations, or silent trials, published from 2015 to 2025. A total of 891 articles were identified, of which 75 met the criteria for inclusion in the final review. We found wide variance in terminology, description and rationale for silent evaluations, leading to substantial heterogeneity in the reported information. Overwhelmingly, the papers reported measurements of area under the curve and similar metrics of technical performance. Far fewer studies reported verification of outputs against an in situ clinical ground truth; when reported, the approaches varied in comprehensiveness. We noted less discussion of sociotechnical components, such as stakeholder engagement and human–computer interaction elements. We conclude that there is an opportunity to bring together diverse evaluative practices (for example, from data science, human factors and other fields) if the silent evaluation phase is to be maximally effective. These gaps mirror challenges in the effective translation of AI tools from computer to bedside and identify opportunities to improve silent evaluation protocols that address key needs.
Source: Nature Health
Authors: Lana Tikhomirov, Carolyn Semmler, Noah Prizant, Srijan Bhasin, Georgia Kenyon, Anton van der Vegt, Lauren Erdman, Nikhil Cherian Kurian, Humphrey Thompson, Lyle J. Palmer, Abdullahi Mohamud, Judy Wawira Gichoya, Seyi Soremekun, Mark P. Sendak, James A. Anderson, Stephen R. Pfohl, Ian Stedman, Daniel Ehrmann, Karin Verspoor, Jethro C. C. Kwong, Lesley-Anne Farmer, Alex John London, Ismail Akrout, Shalmali Joshi, Elena Dicus, Xiaoxuan Liu, Melissa D. McCradden
Institutions: Adelaide University, The University of Adelaide, Duke University, Duke Institute for Health Innovation, Duke Medical Center, The University of Queensland, Cincinnati Children's Hospital Medical Center, University of Cincinnati, Sickkids Research Institute, Emory University, London School of Hygiene & Tropical Medicine, University of Toronto, Hospital for Sick Children, Google (United States), York University, University of Michigan, Michigan Medicine, The University of Melbourne, RMIT University, Carnegie Mellon University, Columbia University, Central Adelaide Local Health Network, University of Birmingham, Women's and Children's Health Network
Language: English
Subfield: Health Informatics
Field: Medicine
Domain: Health Sciences
Open Access status: hybrid
Funder: Hospital Research Foundation
OpenAlex ID: W7129041676