Work

Is the Treatment Weak or the Test Insensitive? Interrogating the Nature of Treatment Effects on Standardized Tests

Year: 2026

Type: article

Abstract: ABSTRACT A general consensus exists that disabilities are harder to treat in older students and improvements are harder to demonstrate on standardized tests. This study used advanced psychometric modeling and computer simulation to examine factors affecting the sensitivity of the Gates‐MacGinitie Reading Test to intervention effects in low ability, middle school students and to examine the possibility that off‐level testing may increase sensitivity to treatment effects. We examine why test sensitivity may vary with student ability and test form used for selection before using computer simulation to examine sensitivity to 10 patterns and sizes of treatment effects. Off‐level testing offered only small improvements to sensitivity. At the same time, findings raised concerns that the lack of effects in real treatment studies may stem from treatments whose effects are either weak or inconsistent and highlighted the failure of intervention research to comprehensively consider psychometric characteristics of measures in designing treatment studies.

Source: Mind Brain and Education

DOI: 10.1111/mbe.70035

Authors: David J. Francis, Paulina A. Kulesz

Institution: University of Houston

Language: English

Topic: Psychometric Methodologies and Testing

Subfield: Management Science and Operations Research

Field: Decision Sciences

Domain: Social Sciences

SDGs: Quality education, Good health and well-being

Open Access status: hybrid

Estimated APC paid: $3,570