Input Prioritization for Testing Neural Networks

Abstract

Deep neural networks (DNNs) are increasingly being adopted for sensing and control functions in a variety of safety and mission-critical systems such as self-driving cars, autonomous air vehicles, medical diagnostics and industrial robotics. Failures of such systems can lead to loss of life or property, which necessitates stringent verification and validation for providing high assurance. Though formal verification approaches are being investigated, testing remains the primary technique for assessing the dependability of such systems. Due to the nature of the tasks handled by DNNs, the cost of obtaining test oracle data-the expected output, a.k.a. label, for a given input-is high, which significantly impacts the amount and quality of testing that can be performed. Thus, prioritizing input data for testing DNNs in meaningful ways to reduce the cost of labeling can go a long way in increasing testing efficacy. This paper proposes using gauges of the DNN's sentiment derived from the computation performed by the model, as a means to identify inputs that are likely to reveal weaknesses. We empirically assessed the efficacy of three such sentiment measures for prioritization-confidence, uncertainty and surprise-and compare their effectiveness in terms of their fault-revealing capability and retraining effectiveness. The results indicate that sentiment measures can effectively flag inputs that expose unacceptable DNN behavior. For MNIST models, the average percentage of inputs correctly flagged ranged from 88% to 94.8%.

DeepCrime: mutation testing of deep learning systems based on real faults

“…By removing such corner cases we expect to get a less discriminative test set, which is less effective in assessing the quality of the model under test. This approach has previously been used for a similar task by Jahangirova & Tonella [22] and for test input prioritisation by Byun et al [13]. In our experiment, for classification systems we build a weak test set by keeping only the test inputs that are predicted with a confidence equal to 1, where confidence is measured as the highest softmax output value.…”

Importance-driven deep learning system testing

“…DeepCT [50] proposes a combinatorial testing approach, while DeepCover [69] adapts MC/DC from traditional software testing and defines adequacy criteria that investigate the changes of successive pairs of layers. Recent research also proposes testing criteria and techniques driven by symbolic execution [31], coverage guided fuzzing [56,76] and metamorphic transformations [72], while other research explores test prioritization [16] and fault localisation [24].…”

Machine Learning Testing: Survey, Landscapes and Horizons

“…These two aspects lead to high test generation costs. Byun et al [148] used DNN metrics like cross entropy, surprisal, and Bayesian uncertainty to prioritise test inputs and experimentally showed that these are good indicators of inputs that expose unacceptable behaviours, which are also useful for retraining.…”