Revisiting the Replication Study Design Used in Computing Education Research
Year: 2026
Type: conference-paper
Study design: Systematic Review
Abstract: Background and Context: Replication studies play an important role in Computing Education Research (CER) by supporting the development of consistent and reliable scientific knowledge. However, prior research indicates that the CER community tends to prioritise novel contributions over replication. A 2019 Systematic Literature Review (SLR) identified only 54 (2.38%) replication studies among 2,269 papers published between 2009 and 2018 across five major CER venues. In response, the Computer Science Education journal released a special issue dedicated to replication studies to encourage greater adoption of this research design. Objectives: This study aims to examine how the landscape of replication research in CER has evolved since 2019. Specifically, we investigate whether the prevalence of replication studies has increased and explore current perceptions and experiences of CER researchers regarding replication. Method: We replicated two prior studies. First, we conducted an updated SLR to identify replication studies published between 2019 and 2025 in the same five CER venues. Second, we replicated a survey of Computing Education researchers to better understand their perceptions, experiences, and challenges related to conducting and publishing replication studies. Findings: Our SLR identified 63 (2.50%) replication studies among 2,516 published papers. While the proportion of replication studies has increased slightly, overall growth remains limited. We observed a shift toward more published replication studies in journals and an increase in authors replicating their own prior work. Survey results indicate that although many researchers engage in replication within their teaching and research practice, they encounter significant challenges when attempting to publish replication studies. Implications: Despite increased discourse around open science and research rigour, the adoption of replication studies in CER has not substantially grown. Our findings offer opportunities for future research to promote replication in CER and to explore how the CER community can encourage researchers to publish replication studies.
Subfield: Computer Science Applications
Field: Computer Science
Domain: Physical Sciences
SDG: Quality education
Open Access status: gold
OpenAlex ID: W7202075618