Work

How do test-takers rate their effort? A comparative analysis of self-report and log file data

Year: 2023

Type: article

Abstract: The present study investigates students' test-taking effort by integrating and comparing traditional self-report questionnaire data and students' test-taking behavior, based on log data analyses. Previous studies have shown that different methods often lead to different results. A computer-based measure of complex problem-solving in uncertain situations was used to minimize the influence of factual knowledge on test performance. K-means cluster analysis was used to build groups of students differing in test-taking effort, resulting in 3 distinct groups. The correlation between students' test-taking effort and test performance proved to be weaker based on the self-reported questionnaire data than on their actual test-taking behavior. Both the self-report questionnaire and the log data showed a decrease in test-taking effort during the test. The number of clicks played the largest role in predicting performance. Results suggest that (1) self-report questionnaire data are not consistent with students' actual test-taking behavior and (2) it's not necessary to make the maximum effort to obtain valid test results, but a certain level of effort is needed.

Source: Learning and Individual Differences

DOI: 10.1016/j.lindif.2023.102340

Authors: Róbert Csányi, Gyöngyvér Molnár

Institution: University of Szeged

Language: English

Topic: Psychometric Methodologies and Testing

Subfield: Management Science and Operations Research

Field: Decision Sciences

Domain: Social Sciences

SDG: Quality education

Open Access status: hybrid

Estimated APC paid: $3,430