Year: 2010
Type: letter
Abstract: The recent editorial on “helping journalists get it right” ( 1 ) was accompanied by an announcement of online resources to assist journalists with reporting about epidemiological studies. We were surprised that the “questions to guide reporting” made no recommendation to start with a question about what is already known about the topic of a research study. Without consideration of prior knowledge on a topic, it is difficult to imagine how any single study could be informative. In addition, the section titled “Could the finding be wrong?” addresses only the potential for residual confounding and assumes that adjustment for confounding could only “weaken” the finding. There is no mention of the possibility that adjustment for confounding could strengthen an association and no mention of the potential influence of other biases, such as information bias and selection bias. The “statistics glossary” for journalists describes the P value as the “probability that an observed effect size is due to chance alone” and the confidence interval as “the range of plausible values around the observed value that will contain the truth 95% of the time.” Neither of these definitions is correct ( 2–4 ). In fact, both definitions have been recognized as important fallacies with serious adverse implications, and warnings against these interpretations have appeared ( 2–4 ). In addition, the statistics glossary promotes statistical significance testing for evaluation of a study's result, but the routine use of statistical significance testing to evaluate epidemiological results is counterproductive ( 2–5 ). It is not clear whether the materials for journalists were peer reviewed, but with such large errors and omissions immediately apparent to us, it seems that either they were not or the peer review was inadequate. These materials are likely to have the opposite of their intended effect; they may help journalists to get it wrong. We applaud any careful effort to communicate research clearly to stakeholders, but these materials do not satisfy a reasonable minimum standard. Incorrect interpretations of frequentist statistics have been a problem in medical science for nearly a century ( 2–7 ). The undersigned authors take responsibility for the accuracy of this letter, and it has been reviewed and endorsed by the signatories at https://sites.google.com/site/editorialsignatories/ , who share the authors’ concern that these problems persist. We all encourage the Journal to suspend access to the site until the materials have been more thoroughly reviewed and revised.
DOI: 10.1093/jnci/djq135
Language: English
Subfield: Statistics and Probability
Field: Mathematics
Domain: Physical Sciences
Open Access status: bronze
OpenAlex ID: W2138823875