Generalizability of consensus regarding standardized letters of evaluation competitiveness: A validity study in a national sample of emergency medicine faculty
Year: 2024
Type: article
Abstract: Background: Standardized letters of evaluation (SLOEs) are an important part of residency recruitment, particularly given the limited availability of other discerning factors in residency applications. While consensus regarding SLOE competitiveness has been studied within a small group of academic faculty, it remains unexplored how a more diverse group of letter readers interpret SLOEs in terms of competitiveness. Methods: A sample of 50 real SLOEs in the new SLOE format (2022 eSLOE 2.0) were selected to match the national rating distribution and anonymized. These SLOEs were ranked in order of competitiveness by 25 faculty members representing diverse demographics, geographic regions, and practice settings. Consensus levels were assessed using previously defined criteria and compared to prior results using a cutoff of ±10% to define a significant difference in consensus levels. Two models were tested to determine their ability to predict consensus rankings: a point-based system and a linear regression model. Results: level of agreement, where consensus was stronger in this study compared to previous results. There is greater consensus among faculty at academic institutions than at community institutions, and years of experience was not correlated with higher consensus. Conclusions: The degree of consensus regarding competitiveness using real SLOEs was similar in this diverse national sample compared to a prior study in a smaller and more homogenous group ranking mock SLOEs. Consensus ranks were predicted with good accuracy using both the point system and the regression model.
Source: AEM Education and Training
DOI: 10.1002/aet2.11049
Institutions: Johns Hopkins University, Johns Hopkins Medicine, Brigham and Women's Hospital, Harvard University, Massachusetts General Hospital, Harvard Affiliated Emergency Medicine Residency, Cooper Medical School of Rowan University, Stanford University, University of Wisconsin–Madison, Beth Israel Deaconess Medical Center, University of Washington, University of Illinois Chicago
Language: English
Field: Medicine
Domain: Health Sciences
SDG: Quality education
Open Access status: bronze
OpenAlex ID: W4405217202