Assessment mode and inconsistent responding on a mixed-worded scale: evidence from TIMSS 2019 across grades and countries
Year: 2025
Type: article
Abstract: Understanding factors that affect inconsistent responding in mixed-worded scales is crucial for ensuring the validity of survey score outcomes. This study investigated whether the assessment mode, defined as participating either in a digital or a paper-based achievement test, was associated with the prevalence of inconsistent responding on a mixed-worded scale on a questionnaire administered immediately after. Data were used from 4th- and 8th-grade students from 16 countries participating in the 2019 Trends in International Mathematics and Science Study (TIMSS). The self-reported mixed-worded scale measured self-concept in mathematics and was administered in paper format to all students. The study employed the mean absolute difference (MAD) and factor mixture analysis (FMA) methods to identify inconsistent respondents. Although cross-cultural variation was observed, the 4th-graders who had taken the computer-based assessment prior to the questionnaire engaged in a slightly higher frequency of inconsistent responses. Among 8th-graders, the difference was smaller and, in several country samples, reversed. Larger prevalence of inconsistent responding was found with the FMA approach. The study emphasizes the need for further research on inconsistent responding across different assessment modes and contexts, suggesting implications for survey methodology and international studies.
Source: Frontiers in Education
Institutions: University of Nicosia, University of Cyprus
Language: English
Subfield: Sociology and Political Science
Field: Social Sciences
Domain: Social Sciences
SDG: Quality education
Open Access status: gold
Estimated APC paid: $2,353
Funder: University of Cyprus
OpenAlex ID: W4411106697