Year: 2019
Type: article
Abstract: All the information one might wish to know from an evaluation literature cannot be conveyed in a simple summary measure. For example, the evidence concerning how supervised consumption sites affect offsite behavior—and hence their population-level cost-effectiveness—is nuanced. Longer-form literature summaries can elucidate such subtle but important ideas. We maintain that the evidence concerning the effectiveness of supervised consumption sites (SCS) is nuanced, and will be more or less satisfying depending on the needs and perspectives of the observer. All three commentaries support that thesis and extend it in useful ways. For example, we distinguished observers based on their function (politician, planner or pioneer) and perspective (academics, advocates and allocators of resources) 1. Frydl 2 and Satel 3 note that people can also care about different goals. Many construe SCS as pure harm reduction, aiming to keep users alive rather than reduce their drug use. Satel suggests probing how users decide to transition to formal treatment—a question that presumes transition to treatment is the goal. Frydl similarly distinguishes between the goals of reducing disorder and infection versus matriculation into evidence-based treatment. A body of evidence could be sufficient for decisionmakers who think of a program as having one goal, but insufficient for those who understand the program's goals differently. Similarly, May et al. 4 make useful suggestions for improving the evidence, such as waiting-list studies and quasi-experimental evaluations that are explicit about logic models and systematically consider and exclude rival hypotheses. Such research is especially needed given that drug markets in parts of North America appear to be transitioning from heroin to more potent opioids, such as fentanyl 5. However, while May et al. 4 may be right that Maryland's Scientific Methods Scale (SMS) could usefully be ported from crime prevention to the addictions field, we would caution—and May et al. would probably agree – that no five-point scale can serve as a sufficient statistic for an evaluation literature. Imagine choosing between two equally costly programs. One scored a 5 (strongest) on the SMS scale because of incontrovertible evidence that it produces a very small positive effect on a subsidiary goal. The other scored a 4 because it had fairly strong evidence of a gigantic effect on the outcome of greatest interest. It would be perfectly rational to fund the second, despite its lower SMS score. The outcome of our paper and the commentaries is that all the information one might wish to know from an evaluation literature cannot be reduced to a simple scale, let alone a ‘thumbs-up’ or ‘thumbs-down’. In particular, it would be a little misleading to describe the literature as giving SCS a ‘thumbs-up’ if that were construed as addressing all questions, concerns or desires for more information. However, this presents its own challenge. Frydl 2 describes larger themes and political discourse within which a program is assessed. Satel 3 is blunter, speaking of the ‘politics of harm reduction research’ and the need for studies to be performed by independent investigators, not (just) those who design or run a program. All researchers might reach the same conclusion if literature assessments were limited to a 1–5 scale, but when assessments include longer narratives, insulating fully against researchers’ biases may be more difficult. Nonetheless, longer-form summaries could permit people to, in Frydl's lovely phrase, ‘part company based on difference more than disagreement’. Our work on this research was supported by internal RAND funds. B.P. and B.K. have no conflicts to declare. J.P.C. has consulted for McKesson, but it was independent of this article.
Source: Addiction
DOI: 10.1111/add.14802
Authors: Jonathan P. Caulkins, Bryce Pardo, Beau Kilmer
Institutions: RAND Corporation, Carnegie Mellon University
Language: English
Topic: HIV, Drug Use, Sexual Risk
Subfield: Epidemiology
Field: Medicine
Domain: Health Sciences
Open Access status: bronze
OpenAlex ID: W2982577340