Predictive analytics using large datasets have long held the promise to improve human decision-making in the most critical domains. Predictive AI is currently deployed to spot cancer and atrial fibrillation before any modern instruments can. Business supply chains can now be scanned both for vulnerabilities and for opportunities to reduce environmental/community harm. And cities may soon use predictive AI to make decisions about what resources are needed where and when to stave off consequences of climate change (e.g., fires, dangerous heat, flooding, etc.) It was supposed to be the same in public safety and policing. However, because our criminal legal system was designed with the goal of producing racially disparate outcomes (among other goals), there are significant moral hazards in equipping police with tools that purport to identify locations, times, and individuals likely to be associated with crime. While much ink has been spilled on both the so-called “denominator” problem in predictive policing and the “endogeneity problem” in both predictive policing and algorithmic risk assessment tools, these approaches tend to focus on the technical obstacles to producing so-called “accurate” models. What the literature rarely engages fully is the more fundamental question: When does math help and when does it hurt vulnerable communities? In this presentation, I intend to present the historical context necessary for making a decision about when to use “fancy math” (in the parlance of several chiefs and activists with whom I have worked) and when more harm than good is likely to result. I will then draw on my own work (both research and praxis) with police and communities over the past 20+ years to discuss patterns of cost and benefit and, I hope, reveal principles capable of putting guardrails around these enormously powerful tools.
