Year: 1997
Type: article
Abstract: ABSTRACT The Natural Language Processing Initiative (NLPI) at ETS currently does research and development for computer‐assisted scoring of natural language constructed‐responses. The purpose of NLPI's work is to develop computer‐assisted methods for scoring so that computer‐administered natural language constructed‐response items can be integrated into ETS' tests which can be scored efficiently with regard to time and cost. Until recently, NLPI's automated scoring efforts were primarily devoted to the development of computer programs used to score short‐answer responses constructed‐responses of up to 15 words. In previous studies, we have developed scoring systems for main idea item types (e.g., GUIDES items) and divergent thinking item types (e.g., the Formulating‐Hypotheses item (FH)). In the current study, we applied similar techniques to those used to score short‐answer responses in order to implement a prototype to automatically score AP Biology essays. This paper reports on the development process involved in building a linguistically‐based, computer scoring prototype for AP Biology essays and the results of the scoring system. The prototype was used to score essays that had been assigned an Excellent rating (9 or 10 points) or a Poor rating (0 ‐ 3 points) by human graders.1 The results reported in the paper illustrate agreement between human rater scores and computer‐based scores. The results of the study are promising and would encourage continued research in automated scoring of essays with highly analytical rubrics.
Source: ETS Research Report Series
Authors: Jill Burstein, Randy Kaplan, Susanne Wolff, Chi Lu
Institution: Educational Testing Service
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: bronze
OpenAlex ID: W2016936751