Methodological quality and reporting practices in gamification research for history education: a critical methodological analysis
Year: 2026
Type: article
Abstract: Gamification has attracted growing research attention as an instructional strategy for history education, yet the methodological rigour of this evidence base has not been systematically evaluated. This study presents a secondary methodological analysis of 74 empirical studies included in a recent systematic review of gamification in history education [15], published between 2004 and 2024 across 27 countries. We introduce a ten-item Methodological Reporting Quality Score (MRQS) to assess the completeness and rigour of reporting practices, and we re-analyse risk-of-bias assessments across three adapted ROBIS domains. The mean MRQS was 4.31 out of 10 (SD = 1.93, median = 4.0), indicating substantial gaps in methodological reporting. The most frequently satisfied criteria were the presence of a theoretical framework (94.6%) and explicit reporting of sample size (83.8%), while the least satisfied were reporting of effect sizes (9.5%), provision of an explicit definition of gamification or game-based learning (14.9%), and use of validated measurement instruments (21.6%). Risk-of-bias analysis confirmed that 83.8% of studies received a high overall rating, with Domain 1 (sampling and selection, 85.1% high) representing the most persistent source of weakness - no study received a favourable rating on sample representativeness - followed by Domain 2 (data collection and analysis, 63.5% high) and Domain 3 (interpretation and reporting, 55.4% high). Studies providing explicit gamification definitions scored higher on the MRQS (5.36 vs. 4.13, p = .065) but showed equivalent risk-of-bias ratings, suggesting that definitional clarity functions as a marker of reporting conscientiousness rather than of design rigour. Comparison of GPT-4o's automated risk-of-bias assessments against human-verified judgments revealed only 37.6% overall agreement (Cohen's κ = -0.04 to 0.21), with the model consistently applying less stringent criteria than human reviewers, while factual metadata extraction across multiple LLMs achieved 82-97% pairwise agreement. These findings indicate that the current evidence base for gamification in history education suffers from pervasive methodological shortcomings that limit the strength of causal inferences and preclude meta-analytic synthesis. We propose the MRQS as a replicable, domain-adapted instrument for assessing reporting quality in educational technology research.
Source: Educational Technology Quarterly
DOI: 10.55056/etq.1126
Authors: Serhii S. Korniienko, Serhiy O. Semerikov
Institution: Kryvyi Rih State Pedagogical University
Language: English
Subfield: Developmental and Educational Psychology
Field: Psychology
Domain: Social Sciences
SDG: Quality education
Open Access status: diamond
OpenAlex ID: W7128963059