Work

Identifying quality responses using an analysis of response times: the RTcutoff function in R

Year: 2024

Type: article

Abstract: Introduction: The present study aims to develop an R function to develop and visualize thresholds that describe the response time of individuals concerning their sample. The function utilizes the cumulative proportion correct (CUMP) approach, to estimate item-specific time threshold, which originated in the work of Guo and his colleagues. Besides the CUMP approach, the present function presents response time profiles on a measure using the mean of the sample and + 1SD times so that it can discern between thoughtful engagement and processing with an item (termed problem-solving behavior) and rapid responding, guessing, and disengagement with the test. The advantage of the CUMP model utilized here is that it simultaneously engages both response time and response correctness to establish thresholds that differentiate engaged from disengaged participants. Methods: = 494) using the Progress in International Reading Literacy Study (PIRLS 2021) international assessment, high and low-achieving individuals that engaged in different behavior patterns were identified and plotted against their sample. Results and conclusion: Results pointed to the importance and necessity of the RTcutoff function to identify variable forms of engagement that have implications for person-score validity but also have implications for test validity and the need to increase measurement precisio.

Source: Frontiers in Psychology

DOI: 10.3389/fpsyg.2024.1479249

Authors: Georgios Sideridis, Mohammed Alghamdi

Institutions: Boston Children's Hospital, Harvard University, National and Kapodistrian University of Athens, King Saud University

Language: English

Topic: Psychometric Methodologies and Testing

Subfield: Management Science and Operations Research

Field: Decision Sciences

Domain: Social Sciences

Open Access status: gold

Estimated APC paid: $3,295

Funder: King Saud University