AI, Opacity, and Personal Autonomy

Abstract

Advancements in machine learning have fuelled the popularity of using AI decision algorithms in procedures such as bail hearings, medical diagnoses and recruitment. Academic articles, policy texts, and popularizing books alike warn that such algorithms tend to be opaque: they do not provide explanations for their outcomes. Building on a causal account of transparency and opacity as well as recent work on the value of causal explanation, I formulate a moral concern for opaque algorithms that is yet to receive a systematic treatment in the literature: when such algorithms are used in life-changing decisions, they can obstruct us from effectively shaping our lives according to our goals and preferences, thus undermining our autonomy. I argue that this concern deserves closer attention as it furnishes the call for transparency in algorithmic decision-making with both new tools and new challenges.

Balancing Privacy and Progress: A Review of Privacy Challenges, Systemic Oversight, and Patient Perceptions in AI-Driven Healthcare

“…Integrating Artificial Intelligence (AI) in healthcare, while offering transformative potential, necessitates a rigorous alignment with ethical standards and medical ethics principles to ensure patient welfare and equitable access. The legal and ethical challenges, particularly concerning data privacy and patient consent, are highlighted by the authors of [28], who stress the importance of robust legal frameworks that are adaptable and responsive to the evolving nature of AI applications. These frameworks must harmonize with regulations such as the General Data Protection Regulation (GDPR) to ensure ethical AI usage in healthcare.…”

“…Private entities' governance of patient data access, usage, and control demands clear guidelines and oversight, addressing the inherent risks of AI, such as errors, biases, and the opacity of algorithmic decision-making processes. To counter the risks of privacy breaches and the potential reidentification of individuals from anonymized datasets, healthcare systems must fortify their defenses with sophisticated computational strategies and ongoing risk assessments [1,28].…”

“…By embedding these measures within the legal and regulatory frameworks that govern AI in healthcare, we can maintain patient and public trust, respect patient dignity and autonomy, and adhere to medical ethics principles. Such an environment is conducive to the responsible development and application of AI, enabling healthcare systems to harness AI's potential to improve patient outcomes, increase care efficiency, and provide equitable access to state-of-the-art medical technology [1,28,29].…”

Ethical issues in research with artificial intelligence systems

“…Another controversial topic is artificial intelligence as a moral advisor or decision-maker [10]. On the other hand, ethical discussions of what we might call the 'ontology' of an AI system are mostly concerned with the problem of opacity [11]. The most powerful systems are those with the most opaque black-box architectures [4].…”

On the Opacity of Deep Neural Networks

“…1 DNNs are also often said to be opaque. 2 Opacity of DNNs causes moral problems, depriving users of full agency (Vaassen, 2022) and obfuscating developer responsibility (Goetze, 2022), but philosophers of science have also discussed the implications of opacity of DNNs. Here, the focus has been whether opacity challenges the application of DNNs to science (Babic et al, 2021;Gunning et al, 2019;Price, 2018;Rudin, 2019;Sullivan, 2022b).…”