Kafkaesque AI? Legal Decision-Making in the Era of Machine Learning

Abstract

Originally published in: Intellectual Property and Technology Law Journal, Vol. 24(2), 251-294 (2020).Artificial Intelligence (“AI”) is already being employed to make critical legal decisions in many countries all over the world. The use of AI in decision-making is a widely debated issue due to allegations of bias, opacity, and lack of accountability. For many, algorithmic decision-making seems obscure, inscrutable, or virtually dystopic. Like in Kafka’s The Trial, the decision-makers are anonymous and cannot be challenged in a discursive manner. This article addresses the question of how AI technology can be used for legal decisionmaking and decision-support without appearing Kafkaesque.First, two types of machine learning algorithms are outlined: both Decision Trees and Artificial Neural Networks are commonly used in decision-making software. The real-world use of those technologies is shown on a few examples. Three types of use-cases are identified, depending on how directly humans are influenced by the decision. To establish criteria for evaluating the use of AI in decision-making, machine ethics, the theory of procedural justice, the rule of law, and the principles of due process are consulted. Subsequently, transparency, fairness, accountability, the right to be heard and the right to notice, as well as dignity and respect are discussed. Furthermore, possible safeguards and potential solutions to tackle existing problems are presented. In conclusion, AI rendering decisions on humans does not have to be Kafkaesque. Many solutions and approaches offer possibilities to not only ameliorate the downsides of current AI technologies, but to enrich and enhance the legal system.

Artificial Intelligence-Powered Criminal Sentencing in Malaysia: A conflict with the rule of law

“…Reiling (Reiling, 2020) suggests that in order to properly utilize AI in the judicial process, the AI in question must be able to provide clarification on how it arrived at the conclusion it did. Justifications for decisions made are important to the notion of the rule of law as they could reveal any shortcoming or errors, such as whether the adjudicators in question have acted arbitrarily or outside its jurisdiction and serve as evidence to support any case against the initial decision (Kemper, 2020). In addition to this, any technological reforms with the promise to improve the justice system must also include the judges.…”

AI and the Industrialization of Surveillance

“…It has been repeatedly established that contemporary AI technologies require very large datasets in order to be developed and become operational (e.g., Chia, Huang, and Koo 2022;Crawford and Paglen 2021;Halevy, Norvig, and Pereira 2009;Kemper 2019;Kotliar 2021;Li 2018;Nawrocki et al 2018;Obermeyer and Emanuel 2016;Virdee 2018;Zuboff 2019: 193). The series of layered statistical techniques utilized in machine learning relies on these data in order to "train" the system to identify links and produce algorithms that predict a certain outcome and to later "validate" these algorithms and their robustness.…”

“…Once looking at the outputs (of both existing tools and of planned outputs for tools under development), it is easy to see that some AI applications have negative implications on human beings (e.g., Davenport and Miller 2022). For instance, AI that is harnessed for predictive policing often tends to be discriminatory (Alikhademi et al 2021;Berk 2021;Kemper 2019;McDaniel and Pease 2021;Zuboff 2019: 386), and AI harnessed for the intensification of the workflow often creates an intolerable work environment (Adams-Prassl 2019; Arnold et al 2018;Moore 2019;Todolí-Signes 2021). Assuming one concurs with the negative-normative assertions of the examples above, it is clear that, in these cases, the contributions to the proliferation of means of surveillance are only adding insult to injury and cannot be justified by their "positive impact.…”

“…Here we turn to the issue of scale. While most technological developments do seem to contribute to such proliferation to some degree, the machine learning data-hungry systems do it on a much larger scale (Chia, Huang and Koo 2022;Crawford and Paglen 2021;Halevy, Norvig, and Pereira 2009;Kemper 2019;Kotliar 2021;Li 2018;Nawrocki et al 2018;Obermeyer and Emanuel 2016;Virdee 2018;Zuboff 2019: 193). Similar to the shift from home/workshop manufacturing to factory manufacturing, here with the move to sophisticated forms of AI, there was a sharp rise in processing power.…”