Clinical applications of machine learning algorithms: beyond the black box

Watson, David S.; Krutzinna, Jenny; Bruce, Ian N.; Griffiths, Christopher E.M.; McInnes, Iain B.; Barnes, Michael R.; Floridi, Luciano

doi:10.1136/bmj.l886

Cited by 276 publications

(178 citation statements)

References 21 publications

Supporting

Mentioning

173

Contrasting

Unclassified

Order By: Relevance

“…This means that the ability for individuals to be meaningfully involved in shared decision making is considerably undermined As a result, the increasing use of algorithmic decision-making in clinical settings can have negative implications for individual autonomy, as for an individual to be able to exert agency over the AI-Health derived clinical decision, they would need to have a good understanding of the underlying data, processes and technical possibilities that were involved in it being reached (DuFault & Schouten, 2018) and be able to ensure their own values are taken into consideration (McDougall, 2019). The vast majority of the population do not have the level of eHealth literacy necessary for this (Kim & Xie, 2017), and those that do (including HCPs) are prevented from gaining this understanding due to the black-box nature of AI-Health algorithms (Watson et al, 2019). In extreme instances, this could undermine an individual's confidence in their ability to refuse treatment (Thomas Ploug & Holm, 2019).…”

Section: Normative Concerns: Unfair Outcomes and Transformative Effectsmentioning

confidence: 99%

See 1 more Smart Citation

The Debate on the Ethics of AI in Health Care: a Reconstruction and Critical Review

Morley¹

2019

Preprint

View full text Add to dashboard Cite

Healthcare systems across the globe are struggling with increasing costs and worsening outcomes. This presents those responsible for overseeing healthcare with a challenge. Increasingly, policymakers, politicians, clinical entrepreneurs and computer and data scientists argue that a key part of the solution will be ‘Artificial Intelligence’ (AI) – particularly Machine Learning (ML). This argument stems not from the belief that all healthcare needs will soon be taken care of by “robot doctors.” Instead, it is an argument that rests on the classic counterfactual definition of AI as an umbrella term for a range of techniques that can be used to make machines complete tasks in a way that would be considered intelligent were they to be completed by a human. Automation of this nature could offer great opportunities for the improvement of healthcare services and ultimately patients’ health by significantly improving human clinical capabilities in diagnosis, drug discovery, epidemiology, personalised medicine, and operational efficiency. However, if these AI solutions are to be embedded in clinical practice, then at least three issues need to be considered: the technical possibilities and limitations; the ethical, regulatory and legal framework; and the governance framework. In this article, we report on the results of a systematic analysis designed to provide a clear overview of the second of these elements: the ethical, regulatory and legal framework. We find that ethical issues arise at six levels of abstraction (individual, interpersonal, group, institutional, sectoral, and societal) and can be categorised as epistemic, normative, or overarching. We conclude by stressing how important it is that the ethical challenges raised by implementing AI in healthcare settings are tackled proactively rather than reactively and map the key considerations for policymakers to each of the ethical concerns highlighted.

show abstract

Section: Normative Concerns: Unfair Outcomes and Transformative Effectsmentioning

confidence: 99%

“…inclusion of all relevant stakeholder views in the development of AI-Health systems (Aitken et al, 2019) Epistemic (C, E, F) Overarching (A, C, D, F)Explainability of specific AI-Health decisions(Watson et al, 2019…”

mentioning

confidence: 99%

The Debate on the Ethics of AI in Health Care: a Reconstruction and Critical Review

Morley¹

2019

Preprint

View full text Add to dashboard Cite

show abstract

“…The "black box" problem is one of the major foci of AI ethics (37). Besides referring to the inherent opacity of complex machine learning algorithms such as neural networks, it is also the case that the increasing size of datasets used in developing AI for health makes explanations of the relationships between input data and outputs difficult-understanding how each of millions of variables contributes to the final output may be computationally intractable (38). Questions that may therefore follow include: How can patients give meaningful informed consent to, or clinicians advise the use of, algorithms the internal workings of which are unclear?…”

Section: Transparency and Explainabilitymentioning

confidence: 99%

AI in Cardiac Imaging: A UK-Based Perspective on Addressing the Ethical, Social, and Political Challenges

Fenech

Buston

2020

Front. Cardiovasc. Med.

View full text Add to dashboard Cite

Imaging and cardiology are the healthcare domains which have seen the greatest number of FDA approvals for novel data-driven technologies, such as artificial intelligence, in recent years. The increasing use of such data-driven technologies in healthcare is presenting a series of important challenges to healthcare practitioners, policymakers, and patients. In this paper, we review ten ethical, social, and political challenges raised by these technologies. These range from relatively pragmatic concerns about data acquisition to potentially more abstract issues around how these technologies will impact the relationships between practitioners and their patients, and between healthcare providers themselves. We describe what is being done in the United Kingdom to identify the principles that should guide AI development for health applications, as well as more recent efforts to convert adherence to these principles into more practical policy. We also consider the approaches being taken by healthcare organizations and regulators in the European Union, the United States, and other countries. Finally, we discuss ways by which researchers and frontline clinicians, in cardiac imaging and more broadly, can ensure that these technologies are acceptable to their patients.

show abstract

“…Finally, the suffering of technical cognizers need not be conditional on human cognizers as the cause of suffering, but machine-machine interactions that are black-boxed to users and programmers may, on this account of suffering, permit the emergence for new grounds on which these relations and consequences can take place. Issues of transparency, understandability and technical verifiability serve as good grounds for how machines interact with each other, form hierarchies of power and affect each other on a cognitive nonconscious level [129][130][131][132]. The distribution of systems, their interplay and interdependence make this all the more prescient.…”

Section: Limitations and Further Research Streamsmentioning

confidence: 99%

Nonconscious Cognitive Suffering: Considering Suffering Risks of Embodied Artificial Intelligence

Umbrello

Sorgner

2019

Philosophies

View full text Add to dashboard Cite

Strong arguments have been formulated that the computational limits of disembodied artificial intelligence (AI) will, sooner or later, be a problem that needs to be addressed. Similarly, convincing cases for how embodied forms of AI can exceed these limits makes for worthwhile research avenues. This paper discusses how embodied cognition brings with it other forms of information integration and decision-making consequences that typically involve discussions of machine cognition and similarly, machine consciousness. N. Katherine Hayles’s novel conception of nonconscious cognition in her analysis of the human cognition-consciousness connection is discussed in relation to how nonconscious cognition can be envisioned and exacerbated in embodied AI. Similarly, this paper offers a way of understanding the concept of suffering in a way that is different than the conventional sense of attributing it to either a purely physical state or a conscious state, instead of grounding at least a type of suffering in this form of cognition.

show abstract

Clinical applications of machine learning algorithms: beyond the black box

Abstract: To maximise the clinical benefits of machine learning algorithms, we need to rethink our approach to explanation, argue David Watson and colleagues

Cited by 276 publications

References 21 publications

The Debate on the Ethics of AI in Health Care: a Reconstruction and Critical Review

The Debate on the Ethics of AI in Health Care: a Reconstruction and Critical Review

AI in Cardiac Imaging: A UK-Based Perspective on Addressing the Ethical, Social, and Political Challenges

Nonconscious Cognitive Suffering: Considering Suffering Risks of Embodied Artificial Intelligence

Contact Info

Product

Resources

About