Methodological alignment and reporting in secondary analyses of DHS, MICS and DHIS2 data: a systematic meta-research study in the East African Community
Year: 2026
Type: preprint
Abstract: Background Secondary analyses of complex surveys and routine health information systems require source-specific methodological safeguards, but published reports do not always distinguish documented analytical incompatibility from incomplete reporting. We estimated how often data-generating structure, inferential target, statistical method and interpretation were aligned in contemporary East African analyses of Demographic and Health Survey (DHS), Multiple Indicator Cluster Survey (MICS) and DHIS2 data. Methods We conducted a systematic methodological study of peer-reviewed single-country analyses published from January 2020 through August 2026. The identified eligible frame contained 466 publications (377 DHS/MICS and 89 DHIS2). A stratified probability sample of 127 publications was coded across four linked domains as aligned, materially misaligned, insufficiently reported or not applicable, with finite-population inference; 26 publications were independently coded for reliability. A prespecified Rwanda illustration compared full survey-design, unweighted and weights-only analyses. Results Fully aligned publications were estimated at 38.7% for DHS/MICS and 40.7% for DHIS2. Material misalignment was 17.3% and 39.2%, while insufficient reporting was 44.0% and 20.1%, respectively. The proportion not fully aligned was 61.3% for DHS/MICS and 59.3% for DHIS2 (difference − 2.0 percentage points, 95% confidence interval − 15.6 to 11.6). Among not fully aligned publications, documented material misalignment accounted for 28.2% of the DHS/MICS total and 66.1% of the DHIS2 total. Overall pre-adjudication agreement was 84.6% (Cohen kappa 0.733; Gwet AC1 0.785). In the 2025 Rwanda DHS, unweighted and weights-only prevalence standard errors were 61.8% and 67.5% of the full-design standard error; the higher-education odds ratio was 8.34 unweighted versus 6.33 with the full survey design. Conclusions Similar overall proportions not fully aligned masked different methodological problems: insufficient observability predominated in DHS/MICS, whereas documented incompatibility contributed more in DHIS2. Methodological assessment and reporting should separate demonstrated misalignment from missing information and apply source-specific safeguards for complex survey design and routine-data provenance, denominators, completeness and temporal structure.
Source: Research Square
Institutions: University of Rwanda, Rwanda Biomedical Center
Language: English
Subfield: Statistics, Probability and Uncertainty
Field: Decision Sciences
Domain: Social Sciences
Keywords: secondary analysis, complex survey design, survey weights, reporting quality, routine health information systems, Demographic and Health Surveys, Multiple Indicator Cluster Surveys, DHIS2, finite population inference, standard errors, East Africa, Rwanda, research reproducibility, odds ratio
Open Access status: green
OpenAlex ID: W7226016951
