Interpretability Through Interrogation: Fairness and Interpretability in the Context of Criminal Sentencing

Abstract

In this paper, I will explore the ways in which algorithms such as those used in criminal sentencing determinations can be biased against demographic subgroups and the attempts that have been made to correct this bias. I will provide policy recommendations for states attempting to use risk assessment tools and provide a framework for addressing future algorithmic bias issues. Specifically, I will explore whether interpretability of machine learning algorithms is necessary and/or sufficient for fairness, and I will argue that it should be one part of a series of ways to audit an algorithm. I will also provide a novel way to audit these algorithms, through reverse engineering the algorithms through their outputs and creating an interactive protocol for model interpretation. This protocol attempts to create fair algorithms through accountability, which can in turn only be attained through interpretability. In the first section, I will describe the motivation, including some background on the criminal justice system and the ways in which algorithms are used, specifically during bail and sentencing determinations. Then, I will provide a review of the literature surrounding not only fairness but also interpretability in machine learning algorithms. Then, I will provide the construction of a protocol, which is an interactive game between a prover and an auditor. I will then apply the theory to the criminal justice system, showing examples of how the protocol can be used in practice. Lastly, I will conclude with policy recommendations and suggestions for further research.