Cochrane's risk of bias tool for non-randomized studies (ROBINS-I) is frequently misapplied: A methodological systematic review
Year: 2021
Type: review
Study design: Systematic Review
Abstract: OBJECTIVES: We aimed to review how 'Risk of Bias In Non-randomized Studies-of Interventions' (ROBINS-I), a Cochrane risk of bias assessment tool, has been used in recent systematic reviews. STUDY DESIGN AND SETTING: Database and citation searches were conducted in March 2020 to identify recently published reviews using ROBINS-I. Reported ROBINS-I assessments and data on how ROBINS-I was used were extracted from each review. Methodological quality of reviews was assessed using AMSTAR 2 ('A MeaSurement Tool to Assess systematic Reviews'). RESULTS: Of 181 hits, 124 reviews were included. Risk of bias was serious/critical in 54% of assessments on average, most commonly due to confounding. Quality of reviews was mostly low, and modifications and incorrect use of ROBINS-I were common, with 20% reviews modifying the rating scale, 20% understating overall risk of bias, and 19% including critical-risk of bias studies in evidence synthesis. Poorly conducted reviews were more likely to report low/moderate risk of bias (predicted probability 57% [95% CI: 47-67] in critically low-quality reviews, 31% [19-46] in high/moderate-quality reviews). CONCLUSION: Low-quality reviews frequently apply ROBINS-I incorrectly, and may thus inappropriately include or give too much weight to uncertain evidence. Readers should be aware that such problems can lead to incorrect conclusions in reviews.
Source: Journal of Clinical Epidemiology
Authors: Erik Igelström(Conceptualization, Formal analysis, Investigation, Methodology, Writing – original draft), Mhairi Campbell(Validation, Writing – review & editing), Peter Craig(Validation, Writing – review & editing), Srinivasa Vittal Katikireddi(Conceptualization, Supervision, Validation, Writing – review & editing)
Language: English
Subfield: Statistics, Probability and Uncertainty
Field: Decision Sciences
Domain: Social Sciences
Open Access status: hybrid
Estimated APC paid: $3,260
Funder: Medical Research Council
Awards: G2720107755, Inequalities in health
OpenAlex ID: W3193705467
