Year: 2025
Type: article
Abstract: Nearly 30 years ago, in response to concerns about the quality of methods used in randomized clinical trials (RCTs), a group of methodologists published the CONSORT (Consolidated Standards of Reporting Trials) guidelines. 1 Although these guidelines are strictly voluntary and pertain only to the reporting of trials, many journals (including this one) now require authors to report their adherence to CONSORT and frequently rate manuscripts that fail to comply less favorably.As a result, the CONSORT guidelines have evolved to become de facto standards for reporting as well as for trial design and analysis.Subsequently, similar reporting guidelines have been developed for nearly every study design, again becoming accepted standards.In 2007, the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) reporting guidelines were issued and endorsed by hundreds of biomedical journals, including this one, which also requires that the STROBE checklist be submitted along with appropriate manuscripts.2 The STROBE checklist, which specifically addresses cohort studies, includes 22 items considered to be essential for good reporting.However, given the wide variety of methods relevant to cohort studies, the criteria are quite general.One key item, for example, is a nonspecific requirement to provide a description of any efforts to address potential sources of bias. 2 Correia and colleagues 3 examined trends in the descriptions of strategies to select confounders in published articles as required by STROBE.The main focus of their work was on clarity of reporting.There is already existing guidance on methodological issues and how to appropriately address confounding bias.4,5 Correia and colleagues identified 623 observational studies meeting their inclusion criteria.A minority of those studies (22 studies [3.5%]) failed to adjust for confounding at all.Others (18 studies [2.9%]) did not specify the selected confounders, or selected confounders without providing a justification (281 studies [45.1%]).The remaining studies provided justifications: 139 (22.3%) selected on the basis of established associations with outcome, 121 (19.9%) selected on the basis of statistical criteria, and 42 (6.7%)selected on the basis of a causal model.3 Thus, approximately 50% of selected articles did not justify their confounder selection method.This is a problem because investigators have numerous options with regard to which variables to choose and the methods for adjustment.Without clearly understanding the rationale for these decisions, it is difficult to interpret the results and assess the validity of conclusions.However, even when authors do provide an adequate description, there remains the concern that they may have engaged in such practices as P value manipulation, also known as P hacking, and generating hypotheses after the results are known, or HARking, to unearth results that may be marginally statistically significant merely as a result of multiple, and possibly spurious, comparisons.Such efforts are typically motivated by the higher likelihood that positive results will be publishable or by heroic efforts to confirm a strongly held hypothesis.Similar concerns related to RCTs led to requirements that they be registered before initiation to be published in a journal that adheres to policies of the International Committee of Medical Journal Editors.There are now 18 clinical trial registries in the World Health Organization Network that meet the requirements of the ICMJE and nearly 100 000 trials have been registered.6 A number of key requirements of a good registry are described by the World Health Organization (eg, having a mechanism in place to ensure the validity of the registered data, being accessible to the public at no charge, and being electronically searchable).Preregistration on 1 or more of these sites makes it apparent when investigators have reported positive findings from analyses that were not contemplated ahead of time, raising concerns of possible manipulation of results +
Source: JAMA Network Open
Authors: Jesse A. Berlin, Stephan D. Fihn
Language: English
Subfield: Statistics, Probability and Uncertainty
Field: Decision Sciences
Domain: Social Sciences
Open Access status: gold
Estimated APC paid: $3,000
OpenAlex ID: W4412654406