Work

Analysis of Log Data From an International Online Educational Assessment System: A Multi-State Survival Modeling Approach to Reaction Time Between and Across Action Sequence

Year: 2025

Type: article

Abstract: With increasingly available computer-based or online assessments, researchers have shown keen interest in analyzing log data to improve our understanding of test takers' problem-solving processes. In this article, we propose a multi-state survival model (MSM) to action sequence data from log files, focusing on modeling test takers' reaction times between actions, in order to investigate which factors and how they influence test takers' transition speed between actions. We specifically identify the key actions that differentiate correct and incorrect answers, compare transition probabilities between these groups, and analyze their distinct problem-solving patterns. Through simulation studies and sensitivity analyses, we evaluate the robustness of our proposed model. We demonstrate the proposed approach using problem-solving items from the Programme for the International Assessment of Adult Competencies (PIAAC).

Source: Psychometrika

DOI: 10.1017/psy.2025.10043

Authors: Jina Park, Ick Hoon Jin, Minjeong Jeon

Institutions: Yonsei University, University of California, Los Angeles

Language: English

Topic: Psychometric Methodologies and Testing

Subfield: Management Science and Operations Research

Field: Decision Sciences

Domain: Social Sciences

Open Access status: hybrid

Estimated APC paid: $3,550

Funders: National Research Foundation, Yonsei University, National Research Foundation of Korea