PROTOCOL: Systematic Review of the Effects of “Pulling Levers” Focused Deterrence Strategies on Crime
Year: 2010
Type: review
Study design: Study Protocol
Abstract: This systematic review will be supported through a $25,000 grant to George Mason University from the National Policing Improvement Agency in the United Kingdom. These funds will support the work of Braga, Weisburd, and a research assistant to conduct the work identified in this review protocol. The Program in Criminal Justice Policy and Management at Harvard University's John F. Kennedy School of Government will also be an intramural source of support for this project. The Program in Criminal Justice will support the research through the provision of office space, computer, phone, fax, and paper supplies. As appropriate, we will seek modest support for the research from external sources such as private foundations and government grant-making agencies. A number of American police departments have been experimenting with new problem-oriented policing frameworks to prevent gang and group-involved violence generally known as the "pulling levers" focused deterrence strategies. These new strategic approaches have shown promising results in the reduction of violence (Braga, Kennedy, and Tita 2002). Pioneered in Boston to halt serious gang violence, the pulling levers framework has been applied in many American cities through federally sponsored violence prevention programs such as the Strategic Alternatives to Community Safety Initiative and Project Safe Neighborhoods (Dalton 2002). In its simplest form, the approach consists of selecting a particular crime problem, such as youth homicide; convening an interagency working group of law enforcement, social-service, and community-based practitioners; conducting research to identify key offenders, groups, and behavior patterns; framing a response to offenders and groups of offenders that uses a varied menu of sanctions ("pulling levers") to stop them from continuing their violent behavior; focusing social services and community resources on targeted offenders and groups to match law enforcement prevention efforts; and directly and repeatedly communicating with offenders to make them understand why they are receiving this special attention (Kennedy 1997, 2006). The pulling levers approach is also consistent with recent theorizing about police innovation, which suggests that approaches that seek to both create more focus in application of crime prevention programs and that expand the tools of policing are likely to be most successful (Weisburd and Eck 2004). Nationally, without the support of a formal evaluation, Boston's Operation Ceasefire pulling levers strategy was hailed as an unprecedented success (see, e.g. Butterfield 1996; Witkin 1997). These claims followed a surprising large decrease in youth homicide after the strategy was fully implemented in mid May 1996. However, more rigorous examinations of youth homicide in Boston soon followed. A U.S. Department of Justice-sponsored evaluation of Operation Ceasefire used a quasi-experimental design to analyze trends in serious violence between 1991 and 1998. The evaluation reported that the intervention was associated with a 63% decrease in monthly number of Boston youth homicides, a 32% decrease in monthly number of shots-fired calls, a 25% decrease in monthly number of gun assaults, and, in one high-risk police district given special attention in the evaluation, a 44% decrease in monthly number of youth gun assault incidents (Braga, Kennedy, Waring, and Piehl 2001). The timing of the "optimal break" in the time series was in the summer months after Ceasefire was implemented (Piehl, Cooper, Braga, and Kennedy 2003). The evaluation also suggested that Boston's significant youth homicide reduction associated with Operation Ceasefire was distinct when compared to youth homicide trends in most major U.S. and New England cities (Braga et al. 2001). Other researchers, however, have observed that some of the decrease in homicide may have occurred without the Ceasefire intervention in place as violence was decreasing in most major U.S. cities. Fagan's (2002) cursory review of gun homicide in Boston and in other Massachusetts cities suggests a general downward trend in gun violence that existed before Operation Ceasefire was implemented. Levitt (2004) analyzed homicide trends over the course of the 1990s and concluded that the impact of innovative policing strategies, such as Operation Ceasefire in Boston and broken windows policing and Compstat in New York, on homicide was limited. Other factors, such as increases in the number of police, the rising prison population, the waning crack-cocaine epidemic, and the legalization of abortion, can account for nearly the entire national decline in homicide, violent crime, and property crime in the 1990s. Using growth-curve analysis to examine predicted homicide trend data for the 95 largest U.S. cities during the 1990s, Rosenfeld and his colleagues (2005) found some evidence of a sharper youth homicide drop in Boston than elsewhere but suggest that the small number of youth homicide incidents precludes strong conclusions about program effectiveness based on their statistical models.1 In his examination of youth homicide trends in Boston, Ludwig (2005) suggested that Ceasefire was associated with a large drop in youth homicide but, given the complexities of analyzing city-level homicide trend data, there remained some uncertainty about the extent of Ceasefire's effect on youth violence in Boston. The National Academies' Panel on Improving Information and Data on Firearms (Wellford, Pepper, and Petrie 2005) concluded that the Ceasefire evaluation was compelling in associating the intervention with the subsequent decline in youth homicide. However, the Panel also suggested that many complex factors affect youth homicide trends and it was difficult to specify the exact relationship between the Ceasefire intervention and subsequent changes in youth offending behaviors. While the DOJ-sponsored evaluation controlled for existing violence trends and certain rival causal factors such as changes in the youth population, drug markets, and employment in Boston, there could be complex interaction effects among these factors not measured by the evaluation that could account for some meaningful portion of the decrease. The evaluation was not a randomized, controlled experiment. Therefore, the non-randomized control group research design cannot rule out these internal threats to the conclusion that Ceasefire was the key factor in the youth homicide decline. The National Academies' Panel also found that the evidence on the effectiveness of the pulling levers focused deterrence strategy in other settings was quite limited (Wellford et al. 2005). The available evidence on the effects of pulling-levers programs in other jurisdictions was scientifically weak. For instance, sudden large decreases in homicide and serious gun violence followed the implementation of pulling levers in Baltimore (Braga, Kennedy, and Tita 2002), Minneapolis (Kennedy and Braga 1998), Stockton (CA) (Wakeling 2003), and High Point (NC) (Coleman, Holton, Olson, Robinson, and Stewart 1999). Unfortunately, these assessments did not use control groups and relied upon simple pre-post measurements of trends in homicide and gun violence. In East Los Angeles, a DOJ-sponsored replication of Operation Ceasefire experienced noteworthy difficulty keeping the social service and community-based partners involved in the interagency collaboration (Tita, Riley, Ridgeway, Grammich, Abrahamse, and Greenwood 2003). However, the law enforcement components of the intervention were fully implemented and focused on two gangs engaged in ongoing violent conflict. The quasi-experimental evaluation revealed that the focused enforcement resulted in significant short-term reductions in violent crime and gang crime in targeted areas relative to matched comparison areas (Tita et al. 2003). Since the publication of the Panel's report, several rigorous evaluations of the effects of pulling levers on gang violence in other jurisdictions have been completed. For instance, a quasi-experimental evaluation of the Indianapolis Violence Reduction Partnership found that the pulling levers strategy was associated with a 42% reduction in homicide in Indianapolis (McGarrell, Chermak, Wilson, and Corsaro 2006). When compared to homicide trends in the nearby cities of Cleveland, Cincinnati, Kansas City, Louisville, and Pittsburgh, the evaluation found that Indianapolis was the only city experiencing a statistically significant decrease in homicide during the study time period. In Chicago, a quasi-experimental evaluation of a Project Safe Neighborhoods gun violence reduction strategy found significant reductions in homicides in treatment neighborhoods relative to control neighborhoods (Papachristos, Meares, and Fagan 2007). The evaluation found that the largest effect was associated with preventive tactics based on the pulling levers strategy, such as offender notification meetings that stress individual deterrence, normative change in offender behavior, and increasing views on legitimacy and procedural justice. Given the growing popularity of pulling levers policing, as well as the conflicting views on its crime prevention value, a systematic review of the empirical evidence on the effects of pulling interventions on crime is necessary to assess the value of this approach to crime prevention. This review will synthesize the existing published and non-published empirical evidence on the effects of pulling levers on crime and will provide a systematic assessment of the preventive value of this approach. Pulling levers focused deterrence strategies represent a specific application of deterrence strategies within a problem-oriented policing framework (Kennedy 2006; Braga 2008; see also Goldstein 1990; Eck and Spelman 1987). In short, pulling levers strategies use the iterative problem-oriented policing process (scanning, analysis, response, and assessment) to frame an interagency response to deter groups of chronic offenders from continuing their ongoing violent conflicts. As such, the proposed review will also include a literature review that sets the practical and theoretical context for the development of pulling levers approaches to crime prevention. Only studies that use comparison group designs involving before and after measures will be eligible for the main analyses of this review. In most pulling levers evaluations (e.g. Braga et al. 2001; McGarrell et al. 2006), the control group experiences routine modern police responses to crime. Control areas usually experience a blend of traditional police responses (e.g., random patrol, rapid response, and ad-hoc investigations) and opportunistic community problem-solving responses. While strategic interventions developed from community policing initiatives may be present in certain control areas, none of the controls engage pulling levers policing strategies to address crime problems. As part of the review, we will also examine and document less rigorous research evaluation evidence (e.g. simple pre-post assessments without control groups). Although we have strong concerns regarding the methodological rigor of such studies, we will identify and analyze them separately from our main analyses. Pulling levers strategies attempt to influence the criminal behavior of individuals through the strategic application of enforcement and social service resources to facilitate desirable behaviors. However, existing reviews of the crime prevention value of pulling levers strategies have noted that published evaluations only report aggregated measures of underlying levels of criminal behaviors in targeted areas (Wellford et al. 2005). For example, in Boston, the pulling levers intervention targeted violent behavior among gang-involved offenders. The evaluators measured behavioral change among gang-involved offenders by examining city-wide trends in aggregated measures of serious violence (Braga et al. 2001). Since most of the existing evidence on the crime prevention value of pulling levers involves aggregated measures of changes in individual behaviors, this review will be limited to area-level analyses of aggregated crime trends. If the search strategies reveal new or unpublished studies that examine individual-level analyses of changes in individual criminal behavior, these studies will be included in our examination as additional descriptive information on the crime prevention mechanisms associated with the pulling levers approach. However, individual-level studies will not be included in the formal systematic review. It is important to note here that the research strategy will yield a diverse set of targeted areas across the identified studies. For example, evaluations of pulling levers policing strategies in Boston and Indianapolis compared the homicide trends in the targeted cities to homicide trends in nonrandomized groups of control cities (Braga et al. 2001; McGarrell et al. 2006). In evaluations of pulling levers programs implemented in Los Angeles and Chicago, however, compared violent crime trends in treatment neighborhoods to violent crime trends in non-randomized groups of control neighborhoods (Tita et al. 2003; Papachristos et al. 2007). This heterogeneity in the units of analysis across studies could have varying and policy-relevant effects on crime prevention outcomes associated with the pulling levers strategies. As such, the review will also classify the types of areas to ensure that the review is measuring similar findings across the potentially diverse set of locations subjected to treatment. Eligible studies must measure the effects of police intervention on officially recorded levels of crime at places. Appropriate measures of crime could include crime incident reports, citizen emergency calls for service, or arrest data. Other outcomes measures such as survey, interview, and victimization measures used by eligible studies to measure program effectiveness will also be coded and analyzed. While all eligible studies must include a crime outcome measure, we will also collect data on community satisfaction measures such as citizen attitudes towards police, fear of crime, and other outcomes. Particular attention was paid to studies that measured crime displacement effects and diffusion of crime control benefit effects. Kennedy (2006) describes a place-based application of pulling levers focused on a disorderly drug market operating in High Point, North Carolina. Policing strategies focused on specific locations have been criticized as resulting in displacement (see Repetto 1976). More recently, academics have observed that crime prevention programs may result in the complete opposite of displacement—that crime control benefits were greater than expected and "spill over" into places beyond the target areas (Clarke and Weisburd 1994). The quality of the methodologies used to measure displacement and diffusion effects, as well as the types of displacement (spatial, temporal, target, modus operandi) examined, was assessed. These different sources will complement each other in the identification of eligible pulling levers policing studies. For example, if an eligible study exists that does not use the one of the search studies or does not appear in one of the on-line databases, contacts with leading researchers and searches of existing bibliographies are likely to discover any such study if it existed. We will also contact researchers involved in previous systematic reviews, such as the recent problem-oriented policing review by Weisburd et al. (2010), to determine whether any pulling levers interventions were identified. All published and unpublished studies will be considered for this review. Each on-line database will be search as far back as possible. However, since pulling levers policing is a very recent development in crime prevention, the search strategies described above should be sufficient to identify all relevant studies. In addition, two existing registers of randomized controlled trials will be consulted. These include (1) the "Registry of Experiments in Criminal Sanctions, 1950–1983 (Weisburd et al. 1990) and (2) the "Social, Psychological, Educational, and Criminological Trials Register" or SPECTR being developed by the United Kingdom Cochrane Centre and the University of Pennsylvania (Turner et al. 2003). The reviewer will screen abstracts and leads to potentially eligible studies and decide which full-text reports should be acquired. Only the full-text papers of titles and abstracts indicating, or potentially indicating, an evaluation of a pulling levers strategy will be obtained. Studies that use randomized controlled designs or quasi-experimental techniques such as matching, statistical controls, comparison groups, and the like will be considered for inclusion in the review. In cases of ambiguity, the full text of the study will be obtained in order to properly determine whether an eligible study design was used. Correlation and observational studies without control groups that examine the effects of pulling levers policing on crime will be noted and appear in catalog form in an appendix to the final report. These descriptive studies will not be included in the formal analysis reporting the findings of the review. Studies meeting the eligibility set forth above will be coded for a range of characteristics related to methodological quality including the definition criteria used to identify the units of analysis, the statistical tests used to determine crime prevention effectiveness, the measurement of displacement, the violation of randomization procedures, from the and the of the by methodological criteria to assess the methodological quality of evaluation studies. These criteria include statistical conclusion internal external and descriptive As and the of the methodological factors on the observed empirical results will be assessed. However, it is important to that eligible studies may not or implementation all implementation and will be not to the value of certain studies study an account of process problems. The with the of a research will information from the full text report on the characteristics of the study a data (see included in A analysis will be on the full text of the report and the data will data on the relevant of this review. These a complete of the used to and identify targeted areas, research design and statistical threats to the research crime outcome and outcome When important information is from available study reports, the researchers will be if to determine if they can that A evaluation of pulling levers policing intervention may provide data on outcome For example, the quasi-experimental evaluation of Boston's Operation Ceasefire intervention an of outcome measures including youth homicide incident data, gun assault incident data, and citizen calls for service data (Braga et al. 2001). studies by and and reported community of the value of Operation Ceasefire in gang violence. Pulling levers interventions could also be targeted at chronic offenders for a of such as violent property and drug The treatment could have varying effects on offending trends and in different crime For cases such as this with findings from the each will be to decide to the findings or to the one that the It is likely that most pulling levers interventions will be to with a specific problem, but some may also target some and report outcomes for these as In these cases the effect for the will be For instance, outcomes are the Braga et al. evaluation the monthly of and in Boston as the outcome However, some studies have outcomes. these separately to regarding statistical of outcomes. As such, we will a of with the criteria of the effect and effect to in an effect for such studies. The strategy will be used for any studies reporting the outcome with different types of data a study the impact of a pulling levers on may use gun homicide gun assault and calls for service as outcome some studies may pulling levers program by one police to specific in areas within a cases will be as one study with and effect for outcomes will be in the as of outcome measures across studies will be out in a and, and As described existing reviews of the effects of pulling levers policing on crime have identified only a studies that use control group evaluation The search strategy and time described may identify a additional studies. The review will of simple descriptive reporting the of studies reporting significant effects on outcome the of the and the of the a the systematic review will upon to assess the effects of the pulling levers interventions on crime. In this each study a for or the effectiveness of treatment. Unfortunately, of results across studies from a number of These to account for the of the studies being (e.g. studies, all being provide more to the of statistical tests (e.g. a large of findings in the provide evidence that the is the and of observed program effects, and, if the statistical of the studies in that of is the of at an conclusion increases as the number of studies on a increases of program effects these by focusing on the and of the effects, not whether the individual effects were statistically and by effect based on the of the effect and the study and 2001). will be used to data from studies. For eligible studies, with data effect will be the measures of effect as suggested in the literature (e.g. see and 2001). effect will be across studies and we will use a such as the for the associated For eligible studies that two groups that have a underlying the effect known as see et al. will be used in our to synthesize This effect measure be for eligible evaluations that an outcome measure, such as homicide for treatment and control areas over and However, a we that several pulling levers policing quasi-experimental such as the Boston Operation Ceasefire evaluation (Braga et al. use time series analyses that trends in serious violence in areas to trends in serious violence in comparison for other rival factors, and existing program are by the change in outcome measures a the and of the pulling levers program over the time The program effect and statistical are compared to similar pre-post in outcome trends in comparison In our proposed systematic review, we will examine the assessment of the treatment relative to comparison in each eligible study to determine whether study to an conclusion regarding the or of an treatment for each eligible we will effect measure that a pre-post that can be compared across and see and to be used in our to synthesize In our proposed we will examine the to assess heterogeneity of effect across studies, it is our that effect is a factor in our that effects from our that pulling levers programs are consistent one to we that there may be across programs that have not been far If the and our of the studies suggests this we will a random effects for all analyses involving effect We also to examine or of pulling levers We are in whether the of the effect across departments or other assess this we will use the to the of analysis (see and for and analysis for or analyses involving As described we will also examine and document less rigorous research evaluation evidence (e.g. simple pre-post assessments without control on the crime prevention effectiveness of pulling levers strategies. the of a recent systematic review by Weisburd et al. (2010), we will report the change for the pre-post studies and the change for all pre-post studies. When more than one outcome is we will these outcomes to create a account for in we will also a change by each study by the of its that crime a The will to a the change of each study and, after each study by the of its we will the This analysis will be from the main analyses described in this and will far less in our conclusions on the crime prevention value of pulling levers strategies. research on crime and outcomes will not be included in this systematic review. on the crime prevention value of pulling levers policing will be included as descriptive information in the review report. If the search strategies reveal a number of studies, the will engage a to in to this review with a of evaluation The for a report the and In with we will this review We like to Wilson, and several for their and on of this protocol. Weisburd does not have any of in conducting a review of the crime prevention effects of pulling levers policing strategies. However, with Braga has a quasi-experimental evaluation that found pulling levers policing to be in serious violence among gang in Boston (see Braga et al. 2001). Although Braga have an towards the effectiveness of pulling levers it may be for to report findings in this review that the findings of his evaluation or related evaluations by his A study must the criteria in order to be each with a or a If the study does not the criteria the The study is a review that is relevant to this (e.g. may have to other studies that are may have range of research research was the study identify the treatment as a pulling levers policing If did the study the crime was targeted for the pulling levers all that were the of the pulling levers all that If the intervention was targeted at the all that the the pulling levers intervention developed based on an analysis of the targeted of analysis was the treatment all that was for the implementation of the the groups were involved in the implementation of the all that key of the pulling levers strategy were identified in the program all that If a strategy was identify the key of the all that If a strategy was were the to the targeted all that did the evaluation about the implementation of the If the process evaluation there were with implementation of the response, these If the process evaluation identified by involved the that were for all that if study was of were study units to treatment or comparison and were controlled that it was to the impact of the intervention or threats to internal were The to the receiving receiving is the exact receiving The to the not receiving treatment receiving were the in this identify any from which the causal were reported that for each a is is the specific outcome recorded on this it the outcome of the this as an outcome of the If of analysis was the of analysis for the research the researchers collect data within the of of data was used to measure the outcome on this If data was specific of data were all that If were types of were all that If were was all that For the units of analysis in this time were for the outcome on this was the in time of the after the the assess the quality of the data the any concerns over the quality of the If the evaluation data to the if the is gang violence, does the evaluation data at whether gang violence If the analysis a in the analysis for this If was a problem, provide many cases were and why were they the above Data more success a of statistical statistically significant between the control and treatment groups or the and treatment a effect If is there data available to an effect of data effect can be and or that the to conclusions about the effectiveness of the intervention in to the outcome being on this about the impact of the the assessment evidence of a displacement of the assessment evidence of a displacement of the that the pulling levers intervention was the there a relationship between the pulling levers intervention and a reduction in the the evaluation the researchers If the of the about about
Source: Campbell Systematic Reviews
DOI: 10.1002/cl2.70
Authors: Anthony A. Braga, David L. Weisburd
Language: English
Subfield: Sociology and Political Science
Field: Social Sciences
Domain: Social Sciences
Open Access status: diamond
OpenAlex ID: W2184096020