ChatGPT as a Learning Tool for Medical Students: Results From a Randomized Controlled Trial
Year: 2025
Type: article
Study designs: Randomized Controlled Trial, Clinical Trial
Abstract: IMPORTANCE: As artificial intelligence (AI) tools like ChatGPT become increasingly widespread in medical education, it is essential to evaluate the effectiveness of AI tools in enhancing students' academic performance and retention compared to traditional educational resources such as lecture materials and textbooks. OBJECTIVE: We aim to determine whether ChatGPT-4.0 improves medical students' short-term academic performance compared to internal institutional resources and publicly available online materials. DESIGN: This study was a single-center, prospective, randomized controlled trial conducted over a two-week period in April 2025. SETTING: The research took place at Georgetown University School of Medicine. PARTICIPANTS: A total of 198 first-year medical students were invited to participate, with 33 students enrolling in the study. Participants (N = 33) were assigned to one of three groups: Group A (ChatGPT-4.0), Group B (external resources, including publicly available online materials such as Google, PubMed, and third-party educational websites, but excluding AI-assisted tools), and Group C (institutional resources such as electronic textbooks and lecture materials). INTERVENTIONS: Individuals (N = 33) were randomly assigned to one of three groups: Group A (ChatGPT-4.0; N = 10, 30.3%), Group B (external resources; N = 12, 36.4%), and Group C (institutional resources; N = 11, 33.3%). Participants completed an initial multiple-choice quiz using their assigned resources, followed by a post-quiz survey. One week later, they retook the same quiz without access to any resources to assess retention. Primary outcome(s) and measure(s): The primary outcome was the initial quiz score. The secondary outcome was the retention score, evaluated through the second quiz without resources. RESULTS: Initial quiz scores (Week 1) were significantly higher in Group A (N = 10, mean = 9.60 ± 0.52) and Group B (N = 12, mean = 9.08 ± 0.79) compared to Group C (N = 11, mean = 6.64 ± 1.57) (p < 0.001). However, retention scores one week later (Week 2) showed no significant differences among the groups: Group A (N = 10, mean = 6.20 ± 1.93), Group B (N = 12, mean = 5.58 ± 2.07), and Group C (N = 11, mean = 4.36 ± 2.01) (p = 0.118). CONCLUSIONS AND RELEVANCE: ChatGPT-4.0 improves short-term academic performance but does not provide a short-term retention advantage over institutional or external online educational resources. These findings demonstrate the potential of AI tools to enhance short-term learning outcomes while emphasizing the need for further research to evaluate their long-term effectiveness in educational settings.
Source: Cureus
DOI: 10.7759/cureus.85767
Authors: Kazi A Kalam, Fadi D Masoud, Adam Muntaser, Raghav Ranga, Xue Geng, Munish Goyal
Language: English
Subfield: Health Informatics
Field: Medicine
Domain: Health Sciences
Open Access status: diamond
OpenAlex ID: W4411210101