Work

Pilot Study on Applying Natural Language Processing Techniques to Classify Student Responses to Open-Ended Problems to Improve Peer Review Assignments

Year: 2015
Type: conference-paper
Abstract: Pilot Study on Applying Natural Language Processing to Classify Student Responses to Open-Ended ProblemsThe use of peer review is an essential part of the engineering design process. The AmericanSociety of Civil Engineers maintains an official policy, formally supporting the use of peerreview in engineering. As an educational tool, peer review can be a valuable way to providestudents feedback without a significant increase in instructor workload. Despite all that iscurrently known about our students, the best mechanism for assigning reviewers to reviewees ina peer review of artifacts is still considered to be blind, random assignment. The underlyingconjecture of this research project is that “there has to be a better way”. Specifically, if amechanism can be identified to accurately predict reviewer quality and reviewee need,complementing matches can be made – assigning high quality reviewers to high need revieweesand vice-versa.This paper represents a follow-up to XXXXX. That study presented the results of an attempt todevelop a classification schema using a large archival database of student work. This papertakes the resulting algorithms produced from that archival dataset and applies them to newstudent work, identifying how well the archival-based classification works on a similar data set.The archival data set came from a 2008 offering of a large (n = 1164 students split into sectionscapped at 120 students) “Introduction to Engineering” course at a large, public, mid-westuniversity. The new data set used the same problem with minimal changes, but was given in2014 to a small (n = 27 students) “Introduction to Engineering” course at a small, private,STEM+Business-only institution in the southeast.The paper will focus on the how the archive-based algorithm was designed, how it was appliedto the 2014 dataset, and the results of that application. The implications of that application onfuture algorithm design will be discussed as well as the next steps for the research.
Language: English
Open Access status: bronze
OpenAlex ID: W2603185421
FWCI:
Cites: 19
Cited by: 1
Related to: 10