Black-box Safety Analysis and Retraining of DNNs based on Feature Extraction and Clustering

Abstract

Deep neural networks (DNNs) have demonstrated superior performance over classical machine learning to support many features in safety-critical systems. Although DNNs are now widely used in such systems (e.g., self driving cars), there is limited progress regarding automated support for functional safety analysis in DNN-based systems. For example, the identification of root causes of errors, to enable both risk analysis and DNN retraining, remains an open problem. In this paper, we propose SAFE, a black-box approach to automatically characterize the root causes of DNN errors. SAFE relies on a transfer learning model pre-trained on ImageNet to extract the features from error-inducing images. It then applies a density-based clustering algorithm to detect arbitrary shaped clusters of images modeling plausible causes of error. Last, clusters are used to effectively retrain and improve the DNN. The black-box nature of SAFE is motivated by our objective not to require changes or even access to the DNN internals to facilitate adoption.Experimental results show the superior ability of SAFE in identifying different root causes of DNN errors based on case studies in the automotive domain. It also yields significant improvements in DNN accuracy after retraining, while saving significant execution time and memory when compared to alternatives. CCS Concepts: • Software and its engineering → Software defect analysis; • Computing methodologies → Machine learning.

DeepGD: A Multi-Objective Black-Box Test Selection Approach for Deep Neural Networks

“…According to Chan et al [44] for traditional software testing, failure-causing inputs tend to be dense and close together. The same insight applies to DNN model testing, since similar mispredicted inputs tend to be due to the same fault [16,43]. Our goal is to devise a test selection method that is not only capable of detecting more mispredicted inputs but also diverse ones in terms of root causes.…”

“…Hence, many papers rely on mispredictions [6,9,20] for test selection evaluation. However, similar to failures in traditional software systems, many mispredicted inputs can be due to the same faults in the DNN model and are therefore redundant [27,43]. When selecting inputs on a limited budget, we should therefore avoid similar or redundant mispredictions as they do not help reveal additional root causes or faults in DNN models.…”

“…Retraining DNN models with unseen inputs carefully selected based on DNN testing results is expected to enhance the model's performance compared to that of the original training set. More specifically, it is highly recommended to retrain DNNs with inputs that have the potential to lead to mispredictions [11,20,43]. Consequently, we aim in this research question to investigate the effectiveness of DeepGD in guiding the retraining of DNN models by selecting inputs that will be labeled and used to improve the model through retraining.…”