Work

Comparison of Peer, Near-Peer, and AI-Assisted Methods to Faculty Grading of SOAP Notes

Year: 2025

Type: article

Abstract: OBJECTIVE: The objective of this study was to determine which approach (peer, near-peer, Chat- Chat-Generative Pretrained Transformer [GPT] Plus, or an inhouse implementation of a GPT-4-based application referred to as GRADES) was closest to faculty grading of Subjective, Objective, Assessment, Plan (SOAP) notes. METHODS: Second-year student pharmacists (n = 83) completed a practice SOAP note. Five methods were used to grade the SOAP notes: faculty, peer, near-peer, ChatGPT Plus, and GRADES. Variability in rubric scores among grading methods was analyzed using Friedman 1-way repeated measures analysis of variance with post hoc testing. Time was tracked for each grading method to evaluate efficiency. RESULTS: The median scores and IQRs for each grading method were as follows: faculty (65% [56%-71%], peer (78% [62%-88%]), near-peer (77% [72%-86%]), ChatGPT Plus (87% [80%-91%]), and GRADES (66% [57%-73%]). Peer-, near-peer, and ChatGPT scores were statistically different from faculty scores. Scores from GRADES were not significantly different from faculty grading. CONCLUSION: The use of an inhouse implementation of GPT-4 (GRADES), with faculty oversight, resulted in rubric scores similar to those of a faculty grader.

Source: American Journal of Pharmaceutical Education

DOI: 10.1016/j.ajpe.2025.101877

Authors: John Pham, Krista L. Donohoe, Dayanjan Wijesinghe, Benjamin Van Tassell, Sarah E. Wheeler, Apryl N. Peddi

Institution: Virginia Commonwealth University

Language: English

Topic: Artificial Intelligence in Healthcare and Education

Subfield: Health Informatics

Field: Medicine

Domain: Health Sciences

SDG: Quality education

Open Access status: gold

Estimated APC paid: $2,400