Worth the weight: An examination of unstructured and structured data in graduate admissions.
Year: 2024
Type: article
Abstract: In graduate admissions, as in many multiattribute decisions, evaluators must judgecandidates from a flood of information, including recommendation letters, personalstatements, grades, and standardized test scores. Some of this information is structured,while some is unstructured. Yet most studies of multiattribute decisions focus on decisionsmade from structured information. This study evaluated how structured and unstructuredinformation is used within graduate admissions decisions. We examined a uniquelycomprehensive dataset of N = 2, 231 graduate applications to the University of Kansas,containing full application packages, demographics, and final admissions decisions for eachapplicant. To make sense of our documents, we applied structural topic modeling (STM), atopic model that allows topic content and prevalence to covary based on other metadata(e.g., department of study). STM allowed us to examine what information the letters andstatements contain, and the relationships between variables like gender and race and thetextual information. We found that most topics in the unstructured data related to specificfields of study. The STMs did not uncover strong differences among applicants regardingrace and gender though the recommendation letters and personal statements forinternational applicants did show some different topic profiles than domestic applicants.We also found that admissions decision-makers behaved as if they prioritized structurednumeric metrics, using unstructured information to check for disqualifications, if at all.However, we found that topics were less reliable than admissions documents, meaning thatadditional ways of using them cannot be completely ruled out. The implications of ourfindings on graduate admissions decisions are discussed.
Source: Decision
DOI: 10.1037/dec0000251
Language: English
Field: Medicine
Domain: Health Sciences
Open Access status: hybrid
Funder: National Science Foundation
OpenAlex ID: W4403945159