Methodological rigour and risk of bias in the publicly available full-text evidence for permanently approved digital health applications (DiGA): a systematic assessment
Year: 2026
Type: article
Study design: Systematic Review
Abstract: Background: In Germany, digital health applications (DiGA) require demonstrated positive care effects for permanent approval. No overarching evaluation has yet assessed the methodological quality and risk of bias of studies underlying the 38 DiGA permanently listed as of January 2025. This study systematically evaluates the methodological rigour of these studies using an evaluation framework. Methods: Evidence quality was systematically assessed in two parts: a dimension-based assessment developed in prior work, and a framework-based assessment using the Oxford Centre for Evidence-Based Medicine (OCEBM) Levels of Evidence and the Revised Cochrane Risk-of-Bias Tool for Randomised Trials (RoB2). Of the 38 permanently approved DiGA, only those with manufacturer-published full-text studies were included. Results: All 29 studies (100%) were randomised controlled trials, classified as OCEBM Level 1b. RoB2 rated 27 (93%) at high risk of bias. Most studies (25/29, 86%) used waitlist control groups and patient-reported outcomes (26/29, 90%). Blinding was inadequate in 17 studies (59%). Four studies (4/29, 14%) showed unbalanced dropout (>10% difference) between arms. Ten of the 38 approved DiGA (26%) lacked published or full-text studies; some available studies were German-language only, limiting generalisability. Only six studies (6/29, 21%) had a balanced sex ratio (within 20%). All studies showed medical benefit, but only four (4/29, 14%) demonstrated patient-relevant structural or procedural improvements. Four studies (4/29, 14%) omitted power analysis, and two (7%) did not reach the required sample size for 80% power. Sample sizes ranged from 56 to 1,245. Follow-up exceeded six months in 19 studies (66%). Conclusions: These findings indicate notable gaps in methodological quality and transparency in the public evidence base for permanently approved DiGA, with many studies at high risk of bias per RoB2. This should be considered alongside structural challenges: blinding is often infeasible for digital, psychological, and behavioural interventions, and patient-reported outcomes are frequently the clinically appropriate measure for the indications studied. These findings concern only published study reports and do not reflect the completeness of the underlying Federal Institute for Drugs and Medical Devices (BfArM) regulatory dossiers. Stronger full-text publication requirements and greater methodological transparency could strengthen the evidence base supporting DiGA reimbursement decisions.
Source: mHealth
Authors: Sebastian Zapp, Ute von Jan, Urs-Vito Albrecht
Language: English
Subfield: Statistics, Probability and Uncertainty
Field: Decision Sciences
Domain: Social Sciences
Open Access status: diamond
OpenAlex ID: W7223173737