Year: 2009
Type: letter
Abstract: The commentary by Boffetta et al. ( 1 ) repeats many familiar warnings about reporting false-positive results from epidemiological studies and calls for increased skepticism when assessing and presenting new results. As most epidemiologists know, cautious skepticism is a hallmark of the scientific approach; indeed, it is sometimes said that epidemiologists have a national flower—the hedge. The authors give examples of what they call false-positive results and summarize studies on serum dichlorodiphenyldichloroethylene (DDE) and breast cancer in their table 1 and depict the temporal trend of risk estimates studies by year of publication in figure 1. Although it is not obvious from the DDE concentrations in table 1, the levels have declined in the United States during the period in which the blood samples were collected. This is a more parsimonious explanation of the trend seen in the figure than the elaborate explanation offered by the authors. Furthermore, the authors dismiss the important findings regarding age of exposure by saying they were “based on subgroup analyses.” Later, the authors speculate that a “cause of false-positive reporting in cancer epidemiology is publication or reporting bias.” They illustrate this problem with a funnel plot (figure 3) and an associated statistical test. Most of the evidence for the existence of this particular bias appears to come from studies of pharmaceutical products. The opposite problem is more likely in occupational and environmental cancer studies because there is a clear incentive for epidemiologists who are employed by industries to publish small negative studies of the potential hazards that their workers face. For example, in the 1980s, a number of negative studies of soft tissue sarcoma among workers who were exposed to phenoxy acid herbicides and their precursors were published by researchers working for the chemical industry. When the National Institute for Occupational Safety and Health, using their legal rights of access to the industries, combined the data from several different corporations that manufactured these chemicals, a large positive study on 2,3,7,8-tetrachlorodibenzo- p -dioxin and exposure to dioxin was finally published. There are also strong incentives on industry-based epidemiologists not to publish positive studies. The experience of the Toxic Substances Control Act of 1976 provides evidence for this. Industries were required to provide the Environmental Protection Agency (EPA) with all data on their chemicals so that the EPA could evaluate the potential hazards of introducing these chemicals into the environment. Fines were levied if it was determined that test data were being withheld. Voluntary reporting by industry was found to be less than anticipated. When EPA granted amnesty (no fines for failure to report) to companies that had not reported new data as required, more than 11 000 never-published scientific studies from 120 companies were sent to the EPA. These included both animal data and human epidemiological studies providing new evidence on the hazardousness of a large number of chemicals ( 2 ). Because publication bias of various types may exist, it is difficult to have much confidence about what kinds of studies are missing from the published record. Under these circumstances, a funnel plot pattern does not provide meaningful evidence. We are confident that the authors are well aware that the false-positive rate is inversely related to another quantity—the false-negative rate. It is disappointing that Boffetta et al. did not acknowledge this fact. We believe that the public's health has been hurt far more by false-negative studies than false-positive ones ( 3–5 ).
DOI: 10.1093/jnci/djn462
Authors: R. W. Clapp, D. Kriebel
Institutions: Boston University, University of Massachusetts Lowell
Language: English
Subfield: Statistics and Probability
Field: Mathematics
Domain: Physical Sciences
Open Access status: bronze
OpenAlex ID: W4235411195