Work

Feasibility of Privacy-preserving Locally Hosted Language Models for Assessing Medical Student Reflective Writing

Year: 2026

Type: article

Abstract: Reflective writing assessment in medical education is labor-intensive and raises data privacy concerns that preclude cloud-based AI solutions. We conducted a feasibility study using a locally-hosted, open-weight large language model (LLM) deployed on institutional servers to assess medical students’ reflective writing. The model evaluated 10 de-identified student narratives using two published rubrics across three independent passes, generating rubric-aligned scores and formative feedback. Scores were consistent across passes and rubrics with no statistically significant differences. Expert faculty rated LLM-generated feedback as meeting most quality criteria (mean 5.48/7). These findings support feasibility of a privacy-preserving, locally-hosted LLM for scalable reflective writing assessment.

Source: Medical Science Educator

DOI: 10.1007/s40670-026-02865-7

Authors: Rick Rejeleene, Jessica N. Byram, Richard M. Frankel, J. Harry Isaacson, Neil B. Mehta

Institutions: Cleveland Clinic Lerner College of Medicine, Case Western Reserve University, Tufts University, Indiana University School of Medicine

Language: English

Topic: Artificial Intelligence in Healthcare and Education

Subfield: Health Informatics

Field: Medicine

Domain: Health Sciences

SDG: Quality education

Open Access status: hybrid

Estimated APC paid: $3,290