Improving Data in Electronic Surveys (IDES)
Year: 2024
Type: report
Abstract: International large-scale assessments administer context questionnaires to students, teachers, and principals to collect information about school, classroom and learning conditions. These questionnaires usually consist of a series of rating type items which often face issues such as social desirability, self-presentation, and acquiescence bias. To address these issues, in the field trial of the International Computer and Information Literacy Study (ICILS), Q-sort was introduced as an alternative question type to assess teachers’ beliefs about teaching. Q-sort is a technique that was initially developed for clinical interviews, requiring respondents to arrange a series of cards according to their preference. In this paper, we use teacher data from 28 eductional systems participating in ICILS 2023 field trial to investigate the differences between two question formats, Q-sort and rating scales both reflecting the same teaching beliefs, using multiple criteria of data quality. We compare the amount of missing data, examine the distribution of responses, item and scale means, and the correlations between the scale scores and teacher characteristics. We observed that the proportion of missing values was higher for Q-sort possibly because the cognitive load is higher for the parallel sorting of a total of 18 items than for the rating items that are answered individually. In addition, we observed more variance in the Q-sort than in the rating version. The ranking of items in Q-sort removes the possibility that respondents can agree equally with all statements and and has thus the potential to reduce acquiescence bias.
DOI: 10.58150/rd_2_4
Language: English
Field: Decision Sciences
Domain: Social Sciences
SDG: Quality education
Open Access status: green
Award: G6716689930
OpenAlex ID: W7206216664