Identifying signals of interest when screening for drug‐outcome associations in health care data
Year: 2018
Type: editorial
Abstract: Postapproval drug safety studies often test a prespecified hypothesis of association between a drug exposure and an outcome. A testable hypothesis may come from spontaneous reporting schemes, published case reports, analogy to chemically similar drugs, worrisome results from clinical trials or even from speculation and anecdote. Large, linked health data resources have become the standard venue for such hypothesis-driven drug safety research in Europe and North America. The use of these health care data as a resource for generating hypotheses is more controversial. Some initiatives to screen health care data for drug safety signals have been around for a long time, such as the case–control surveillance of Boston University's Drug Epidemiology Unit 1 and the Kaiser Permanente drug-cancer screening programme 2, while newer efforts include screening for drug-cancer associations in Denmark 3 and screening across all possible outcomes for selected medical products in the US Food and Drug Administration's Sentinel System 4, 5. A key challenge for screening initiatives is differentiation of potential safety signals from background noise. Here, we (i) briefly introduce an example of such a large-scale screening endeavour and (ii) discuss different approaches that can be used to filter the signals to separate out a tractable number that merit subsequent studies. We recently reported on a screening study in Denmark based on all prescriptions and secondary care contacts over an 18-year period for Danes born 1950 or earlier 6. The study used a sequence symmetry analysis (SSA) design 7. The analytical unit for the SSA is the sequence of, e.g. new drug use and a new disease in an individual experiencing both. If no association exists between the drug and the disease, then one would expect to see equally many drug ➔ disease and disease ➔ drug sequences. If, on the other hand, the drug is associated with an increased risk of the disease, then more drug ➔ disease than disease ➔ drug sequences would be expected. The SSA has the advantages of being computationally efficient while being robust to confounders that are stable over time. In the screening study 6, more than 200 billion individual sequences of events were processed, looking for drug ➔ drug or drug ➔ diagnosis sequences that occurred more frequently than explained by chance. The rationale for looking at drug ➔ drug sequences in addition to drug–disease sequences was that some drugs might cause adverse effects that would not lead to hospital contact but might prompt the use of a different drug. For example, starting treatment with opioids might cause constipation and therefore lead to use of laxatives. In all, 43 575 drug ➔ drug or drug ➔ disease pairs showed associations in the main analysis. These were ranked according to the number of outcomes that could potentially be attributed to the triggering drug, thus assuming that – all other thing being equal – highest-ranked associations would represent those with the largest potential public health impact. We further inspected the top 200 drug ➔ drug and 200 drug ➔ disease associations and classified them as unknown associations, known adverse drug reactions and associations indicative of sound clinical practice rather than drug effects. For example, use of paracetamol was associated with later use of opioids, reflecting rational use of first-line before second-line drugs for pain management. In screening studies, signals can arise from three sources: true causal relations, bias (e.g. uncontrolled confounding) or chance. One might initially try to filter out associations that are likely due to bias or chance by further interrogating that data that gave rise to the signals, either through automated decision rules or by manual inspection. Also, application of external biological, pharmacological and epidemiological knowledge can help to further reduce signals that are likely of noncausal origin as well as to prioritize remaining signals as for which should be followed up in subsequent studies. We present several approaches that can be considered, although not all are relevant to all settings. First, analyses using epidemiological designs different from the one used to generate the signal can help to evaluate the robustness of the signal to different sets of assumptions. For example, if screening was performed using a case–control design, self-controlled designs 8, such as the case–crossover design 9 or the SSA described above 7, could be used to assess the robustness of the signals to methods that have different strengths and assumptions. The self-controlled designs have as an inherent strength the ability to control for confounding by factors that do not vary over time – regardless of whether they are measured or unmeasured. However, careful interpretation is important when considering results from multiple designs as truly causal relationships may not be detectable by every design. For example, a self-controlled design might be useful for detecting abrupt outcomes (e.g. bleeding events) but is likely of little help in identifying causal drug-cancer associations 10. Second, assessment of dose–response patterns can in some cases inform whether a given drug-outcome association supports a causal link. As an example, in drug-cancer screening, it seems reasonable to expect that, if causally related, higher cumulative drug exposure would confer greater risk of an outcome. While confounding by indication can belie apparent dose–response relationships – e.g. greater severity of an underlying condition may predispose to higher outcome risk and warrant higher treatment doses – the absence of an apparent dose–response relationship may indicate a noncausal association. Third, assessment of temporality can also add to the understanding of a potential drug-outcome association. If a short-term exposure (e.g. vaccines) leads to an increased risk that is clustered in time, this may strengthen our belief in a causal association 11, whereas a uniformly increased risk might suggest confounding due to differences in characteristics among those who use versus do not use the drug rather than a true effect of the drug itself. Assessment of temporality might even extend to assessing the occurrence of outcomes before exposure, such as when using the symmetry design 7, 8. If the outcome frequency is elevated both before and after drug exposure, confounding is likely at play. Similarly, if a drug is unlikely to have an immediate effect on an outcome (e.g. cancer), early increases in risk may point to reverse causation 12 or other differences in underlying patient characteristics between those who use the drug and those who do not. Fourth, one should consider biological plausibility, e.g. whether the drug is known to affect the organ system in question or whether the outcome can meaningfully be correlated to known properties of the drug. As an example, the knowledge that hydrochlorothiazide is known to possess photosensitizing properties 13 strengthened the belief that an observed association between hydrochlorothiazide and cancer of the lip could be causal 14. While plausibility can to some extent be formalized in systems pharmacology 15, prioritization will need to rely on human synthesis of current, imperfect knowledge for some time to come 16. As such, care should be taken so as not to dismiss otherwise robust signals on this criterion alone. Fifth, it is necessary to further consider and quantify potential confounding even if a filter has already been applied to separate out signals that are clearly due to bias. Confounding refers to the mixing of effects, that is, that an apparent association of an outcome to a drug is in fact due to other characteristics (measured or unmeasured) among the users of that drug and not the effect of the drug use itself. As an example, the Danish drug-cancer screening found a strong association between use of drugs for obstructive airway diseases and lung cancer that even displayed a dose–response pattern 3. As smoking causes both lung cancer and obstructive airway disease, the association is likely due to confounding. Dismissing such signals that stem from structural, but noncausal, relations between drugs and outcomes cannot be easily automated. All observations of association between a drug and outcome are subject to potential confounding, but to different degrees. If the data are readily available, additional more rigorous confounding adjustment could be applied. However, often the relevant confounding factor might either be unknown or at least unmeasured. Quantitative bias analyses 17 could provide insight into whether the observed association could plausibly be attributed to bias, such as by comparing estimates of the required magnitude of confounding to the strength of known risk factors for the outcome in question. Finally, to prioritize remaining signals for subsequent studies, consideration should be given to the potential disease burden that can be attributed to a drug-outcome association, should it be causal, as we did in the SSA example described above. The burden is a function of the extent of use of the drug, the baseline incidence of the outcome, the strength of the drug-outcome association and the seriousness of the outcome. Unfortunately, the latter is somewhat subjective, and patient preferences may not always coincide with those of investigators or regulators trying to prioritize signals. As such, there is no widely accepted method for weighting outcomes by seriousness. By conflating effect size and statistical precision, P values may be a useful tool for prioritizing signals in the setting of screening alongside these other considerations but should not be interpreted with the same thresholds for statistical significance as applied to studies evaluating a single hypothesis. Signals that remain of interest after initial evaluation and are of high priority will need to undergo further evaluation. This might include further checks of robustness within the same database that generated the signal 18, such as more refined analysis (tailored confounder adjustment, subgroup analyses, etc.) or investigation of orthogonal hypotheses 19 that are biologically related to but statistically independent from the initial screening analyses. When available, further analyses should leverage separate data sources that complement the dataset that gave rise to the signal, such as by providing data on potentially important confounders that could not be accounted for in the original data source. The use of large linked health data resources for high-throughput screening activities holds promise. However, considerable methodological work is needed to fully characterize the potential for such studies to contribute to the postmarketing surveillance of both established and newly marketed medications. This pertains not only to the filtering of signals as discussed here but also to the design and conduct of the screening studies themselves. While multiple semi-automated systems have been established, thoughtful clinical and epidemiological input will always be needed to assure the quality of such studies. Time will tell exactly how these approaches will fit into the evolving pharmacovigilance armamentarium. S.V.W. and J.J.G. receive salary support for work in the Sentinel System, which is funded by the US Food and Drug Administration. J.J.G. is a PI of grants from Eli Lilly and Company and Novartis Pharmaceuticals Corporation to the Brigham and Women's Hospital and is a consultant to Aetion, Inc., and Optum, Inc., all for unrelated work. J.H. and A.P. have stated explicitly that they have no competing interests in connection with this article. The authors wish to thank Alexander M. Walker for valuable input and discussions both before and during the preparation of this manuscript.
DOI: 10.1111/bcp.13634
Language: English
Subfield: Toxicology
Domain: Life Sciences
Open Access status: green
OpenAlex ID: W2805480072