An XAI method for convolutional neural networks in self-driving cars

Abstract

eXplainable Artificial Intelligence (XAI) is a new trend of machine learning. Machine learning models are used to predict or decide something, and they derive output based on a large volume of data set. Here, the problem is that it is hard to know why such prediction was derived, especially when using deep learning models. It makes the models unreliable in the case of reliability-critical applications. So, it is required to explain how they derived such output. It is a reliability-critical application for self-driving cars because the mistakes made by the computers inside them can lead to critical accidents. So, it is necessary to adopt XAI models in this field. In this paper, we propose an XAI method based on computing and explaining the difference of the output values of the neurons in the last hidden layer of convolutional neural networks. First, we input the original image and some modified images of it. Then we derive output values for each image and compare these values. Then, we introduce the Sensitivity Analysis technique to explain which parts of the original image are needed to distinguish the category. In detail, we divide the image into several parts and fill these parts with shades. First, we compute the influence value on the vector indicating the last hidden layer of the model for each of these parts. Then we draw shades whose darkness is in proportion to the influence values. The experimental results show that our approach for XAI in self-driving cars finds the parts needed to distinguish the category of these images accurately.

Ensemble learning based transmission line fault classification using phasor measurement unit (PMU) data with explainable AI (XAI)

“…Hence, optimal parameters of these classifiers that ensure consistent performance have been used to construct the final classification model. The performance of these consistently effectuating models has been cross-verified through explainable AI [ 50 , 52 , 60 ], which increased the dependability of the classification model.…”

A Systematic Literature Review of Explainable Artificial Intelligence (XAI) in Software Engineering (SE)

“…However, in the past, research initiatives in various fields have been launched using XAI approaches, bringing insightful insights and advancements. These XAI techniques have previously been used in a wide range of different domains, including Healthcare [13][14][15][16][17], Finance [19][20][21][22][23], transportation [12], legal [18], and others. XAI is a developing area of research within the SE domain, with relatively minimal progress.…”

“…This misclassification can endanger people's lives. As a result, only an Explainable system can resolve the ambiguity in such situations [12]. 5.2 Healthcare XAI is becoming increasingly important in the Healthcare domain.…”

Identifying real-world problems with automated vehicles by detecting behavioral differences in steering movements between the human driver and machine

“…However, there remain many unresolved issues in the known approaches, for which no solution exists yet. The purpose of this research work is to contribute to the certification process, as many new challenges are arising due to the increasing complexity, advancing automation, and rapid updating of software systems [3].…”