Person:

Winship, Christopher

Research Summary This article reviews the causal turn in the social sciences and accompanying efforts by criminologists to make policy claims more credible. Although there has been much progress in techniques for the estimation of causal effects, we find that the link between evidence and valid policy implications remains elusive. Drawing on criminological theory and research insights from disciplines such as sociology, economics, and statistics, we assess principles and strategies for informing policy in a causally uncertain world. We identify three distinct domains of inquiry that form a part of the translational process from evidence to policy and that complicate the straightforward exportation of causal effects to policy recommendations: (a) mechanisms and causal pathways, (b) effect heterogeneity, and (c) contextualization. We elaborate these three concepts by examining research on broken windows theory, policing, video games and violence, the Moving to Opportunity voucher experiment, incarceration, and especially the rich set of experimental studies on domestic violence that originated in Minneapolis, MN in the early 1980s. We also articulate a set of conceptual tools for advancing the goal of policy translation and offer recommendations for how what we call “policy graphs”—causal graphs used to analyze the policy implications of a system of causal relations—can potentially integrate the theoretical and policy arms of criminology.

Policy Implications Evidence, even if causal, does not necessarily inform policy. In fact, the question of “what works,” the focus of the growing evidence-based movement in criminology, turns out to be a different question than, “what will work?” Evidence-based policy research must therefore be concerned with much more than providing policymakers with research on causal effects, however precisely measured. The implication is that we must separate criminology’s increasing focus on causality from its policy turn and formally recognize that the latter requires a different standard of theory and evidence than does the former. In particular, criminologists interested in making policy claims must ask hard questions about the potential mechanisms through which a treatment influences an outcome, heterogeneous effects across people and time, contextual variations, and all of the real-world phenomena to which these challenges give rise—such as unintended consequences, policies that change incentive and opportunity structures, and the scale at which policies change in meaning. Theoretically guided causal graphs enhance this goal and help inform policy in a causally uncertain world. Translational criminology is ultimately a process that entails the constant interplay of theory, research, and practice.

The collection of large-scale administrative records in electronic form by many cities provides a new opportunity for the measurement and longitudinal tracking of neighborhood characteristics, but one that requires novel methodologies that convert such data into research-relevant measures. The authors illustrate these challenges by developing measures of “broken windows” from Boston’s constituent relationship management (CRM) system (aka 311 hotline). A 16-month archive of the CRM database contains more than 300,000 address-based requests for city services, many of which reference physical incivilities (e.g., graffiti removal). The authors carry out three ecometric analyses, each building on the previous one. Analysis 1 examines the content of the measure, identifying 28 items that constitute two independent constructs, private neglect and public denigration. Analysis 2 assesses the validity of the measure by using investigator-initiated neighborhood audits to examine the “civic response rate” across neighborhoods. Indicators of civic response were then extracted from the CRM database so that measurement adjustments could be automated. These adjustments were calibrated against measures of litter from the objective audits. Analysis 3 examines the reliability of the composite measure of physical disorder at different spatiotemporal windows, finding that census tracts can be measured at two-month intervals and census block groups at six-month intervals. The final measures are highly detailed, can be tracked longitudinally, and are virtually costless. This framework thus provides an example of how new forms of large-scale administrative data can yield ecometric measurement for urban science while illustrating the methodological challenges that must be addressed.