The Role of Explainability in Assuring Safety of Machine Learning in Healthcare

Jia, Yan; McDermid, John; Lawton, Tom; Habli, Ibrahim

doi:10.1109/tetc.2022.3171314

Cited by 37 publications

(17 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although AI models have achieved human-like performance, their use is still limited, partly because they are seen as a black box [23,24]. As presented by Jia et al [25], the explainability in an emerging issue, particularly in ML-based healthcare systems. The problem with the use of AI-based tools in medicine continues to be the lack of confidence of medical professionals in such solutions and the perception that they lack the 'intuition' that experienced professionals possess [26,27].…”

Section: Discussionmentioning

confidence: 99%

Influence of augmentation on the performance of double ResNet-based model for chest X-rays classification

Kłoska¹,

Tarczewska²,

Giełczyk³

et al. 2023

Pol J Radiol

View full text Add to dashboard Cite

Purpose A pandemic disease elicited by the SARS-CoV-2 virus has become a serious health issue due to infecting millions of people all over the world. Recent publications prove that artificial intelligence (AI) can be used for medical diagnosis purposes, including interpretation of X-ray images. X-ray scanning is relatively cheap, and scan processing is not computationally demanding. Material and methods In our experiment a baseline transfer learning schema of processing of lung X-ray images, including augmentation, in order to detect COVID-19 symptoms was implemented. Seven different scenarios of augmentation were proposed. The model was trained on a dataset consisting of more than 30,000 X-ray images. Results The obtained model was evaluated using real images from a Polish hospital, with the use of standard metrics, and it achieved accuracy = 0.9839, precision = 0.9697, recall = 1.0000, and F1-score = 0.9846. Conclusions Our experiment proved that augmentations and masking could be important steps of data pre-processing and could contribute to improvement of the evaluation metrics. Because medical professionals often tend to lack confidence in AI-based tools, we have designed the proposed model so that its results would be explainable and could play a supporting role for radiology specialists in their work.

show abstract

Section: Discussionmentioning

confidence: 99%

Influence of augmentation on the performance of double ResNet-based model for chest X-rays classification

Kłoska¹,

Tarczewska²,

Giełczyk³

et al. 2023

Pol J Radiol

View full text Add to dashboard Cite

show abstract

“…Another operational mitigation is to show the clinician similar patients from the TD to the one which the predictor has been applied to (i.e., prototypical examples as described in [10]). This would allow the clinician to review similar cases, their progression, and provides context to a particular prediction.…”

Section: Discussionmentioning

confidence: 99%

The Impact of Training Data Shortfalls on Safety of AI-Based Clinical Decision Support Systems

Ryan Conmy,

Ozturk,

Lawton

et al. 2023

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Decision support systems with Artificial intelligence (AI) and specifically Machine Learning (ML) components present many challenges when assuring trust in operational performance, particularly in a safety-critical domain such as healthcare. During operation the Human in/on The Loop (HTL) may need assistance in determining when to trust the ML output and when to override it, particularly to prevent hazardous situations. In this paper, we consider how issues with training data shortfalls can cause varying safety performance in ML. We present a case study using an ML-based clinical decision support system for Type-2 diabetes related co-morbidity prediction (DCP). The DCP ML component is trained using real patient data, but the data was taken from a very large live database gathered over many years, and the records vary in distribution and completeness. Research developing similar clinical predictor systems describe different methods to compensate for training data shortfalls, but concentrate only on fixing the data to maximise the ML performance without considering a system safety perspective. This means the impact of the ML's varying performance is not fully understood at the system level. Further, methods such as data imputation can introduce a further risk of bias which is not addressed. This paper combines the use of ML data shortfall compensation measures with exploratory safety analysis to ensure all means of reducing risk are considered. We demonstrate that together these provide a richer picture allowing more effective identification and mitigation of risks from training data shortfalls.

show abstract

“…This can both lower trust and safety of these systems. [19][20][21] Another crucial issue to consider is the data. As AI-based are being trained on existing data, the model can only be as good as the data used for distilling the information into the system.…”

Section: Network Examples Of DL Are Convolution Neuralmentioning

confidence: 99%

Introduction to artificial intelligence and deep learning using interactive electronic programming notebooks

Menke

Homberg

Koch

2023

Archiv der Pharmazie

View full text Add to dashboard Cite

Artificial intelligence (AI), or deep learning (DL), approaches have already found their way into our everyday lives. Furthermore, these methods are a central part of research in the life and natural sciences and have been applied in the form of machine learning for decades. In pharmaceutical and medicinal chemistry, and in computer-aided drug discovery, current developments are also changing the way drugs are developed. It is essential to familiarize students with AI methods already during their studies and prepare them for future tasks and challenges. We developed a set of interactive learning materials based on cheminformatics examples that can be used to establish such introductory AI courses in the life and natural sciences.These interactive notebooks are easily accessible without the need for installation, and no prior programming knowledge is required. Through these notebooks, students can easily study how AI/DL works and how these methods can be applied. This knowledge will foster a general competence when interacting with and evaluating future DL applications later in their career. The materials are freely available and publicly accessible through a GitHub repository in German and English.

show abstract

The Role of Explainability in Assuring Safety of Machine Learning in Healthcare

Cited by 37 publications

References 42 publications

Influence of augmentation on the performance of double ResNet-based model for chest X-rays classification

Influence of augmentation on the performance of double ResNet-based model for chest X-rays classification

The Impact of Training Data Shortfalls on Safety of AI-Based Clinical Decision Support Systems

Introduction to artificial intelligence and deep learning using interactive electronic programming notebooks

Contact Info

Product

Resources

About