Uncertainty, risk and the use of algorithms in policy decisions: a case study on criminal justice in the USA

Abstract

Algorithms are increasingly used in different domains of public policy. They help humans to profile unemployed, support administrations to detect tax fraud and give recidivism risk scores that judges or criminal justice managers take into account when they make bail decisions. In recent years, critics have increasingly pointed to ethical challenges of these tools and emphasized problems of discrimination, opaqueness or accountability, and computer scientists have proposed technical solutions to these issues. In contrast to these important debates, the literature on how these tools are implemented in the actual everyday decision-making process has remained cursory. This is problematic because the consequences of ADM systems are at least as dependent on the implementation in an actual decision-making context as on their technical features. In this study, we show how the introduction of risk assessment tools in the criminal justice sector on the local level in the USA has deeply transformed the decision-making process. We argue that this is mainly due to the fact that the evidence generated by the algorithm introduces a notion of statistical prediction to a situation which was dominated by fundamental uncertainty about the outcome before. While this expectation is supported by the case study evidence, the possibility to shift blame to the algorithm does seem much less important to the criminal justice actors.

Exploring the roles of trust and social group preference on the legitimacy of algorithmic decision-making vs. human decision-making for allocating COVID-19 vaccinations

“…Such anecdotal evidence is in line with recent research which suggests that the implementation of ADM in public administration has been far from smooth (Hartmann and Wenzelburger 2021 ). Even the most well-intentioned ADM may falsely discriminate against certain groups; such systems often violate the “established weighting of relevant ethical concerns in a given context” (Heinrichs 2021 , 1).…”

Using automated decision-making (ADM) to allocate Covid-19 vaccinations? Exploring the roles of trust and social group preference on the legitimacy of ADM vs. human decision-making

“…Such anecdotal evidence is in line with recent research that suggests that the implementation of ADM in public administration is far from unproblematic (Hartmann and Wenzelburger 2021). Even the most well-intentioned ADM may falsely discriminate against certain groups, i.e., such systems often violate the "established weighting of relevant ethical concerns in a given context" (Heinrichs 2021, 1).…”

Especificando la responsabilidad algorítmica

“…El algoritmo estereotipa al usuario. Por ejemplo: clasifica a una mujer exenta del riesgo de violencia doméstica; sanciona o libera a un infractor por alta o baja probabilidad de cometer delitos (Hartmann and Wenzelburger, 2021); define a sujetos como inapropiados para una ampliación de crédito o niega una beca.…”