Work

Comment on “Evidence Synthesis of Observational Studies in Environmental Health: Lessons Learned from a Systematic Review on Traffic-Related Air Pollution”

Year: 2024

Type: review

Abstract: We read with interest the discussion by Boogaard et al. 1 of their experience using the OHAT systematic review framework 2 to assess the level of confidence in the epidemiological literature included in their systematic review on traffic-related air pollution.OHAT is the abbreviation for Office of Health Assessment and Translation, which has become the Health Assessment and Translation group in the Integrative Health Assessment Branch of the Division of Translational Toxicology at the National Institute of Environmental Health Sciences.The authors concluded that the OHAT approach, and other frameworks using the Grading of Recommendations, Assessment, Development, and Evaluations, require substantial modification to better align with environmental health questions.We appreciate their interest in both using and improving the OHAT approach.However, they did not take full advantage of the flexibility built into the approach, and their recommendations reflect several inaccuracies and misinterpretations such that most of their suggested modifications are already part of the method.Boogaard et al. state that when translating confidence ratings to level-of-evidence conclusions, the OHAT approach does not consider four "important relevant factors" (study number and size, direction and magnitude of associations, consistency of results, generalizability) or three "additional features" (sufficient time between exposure and outcome, adequate exposure contrast, time windows for exposure and outcome).We agree that these issues are important to evidence integration, which is why each is considered in Step 5 of the OHAT methodology (except for "adequate exposure contrast," which is considered in Step 4). 2 The authors suggest considering these issues a second time in Step 6; however, double counting any of these factors would bias the evidence assessment.Additionally, the OHAT approach anticipates that additional project-specific considerations may be warranted for developing confidence ratings.Throughout the OHAT handbook, an "other" category is already included to accommodate potential issues.In setting initial confidence for bodies of evidence, the authors appear to have correctly followed the OHAT approach; however, they inaccurately stated that their method differed from the OHAT guidance.When determining final study confidence, the authors reported that they did not use two of the grading factors described in the handbook-indirectness or large magnitude of effect-thereby skipping several considerations that they later cite as missing from the OHAT approach.It is not clear why the authors skipped those factors, but it is inappropriate to omit steps

Source: Environmental Health Perspectives

DOI: 10.1289/ehp15644