Black-Box Testing of Deep Neural Networks Through Test Case Diversity

Abstract

Deep Neural Networks (DNNs) have been extensively used in many areas including image processing, medical diagnostics, and autonomous driving. However, DNNs can exhibit erroneous behaviours that may lead to critical errors, especially when used in safety-critical systems. Inspired by testing techniques for traditional software systems, researchers have proposed neuron coverage criteria, as an analogy to source code coverage, to guide the testing of DNN models. Despite very active research on DNN coverage, several recent studies have questioned the usefulness of such criteria in guiding DNN testing. Further, from a practical standpoint, these criteria are white-box as they require access to the internals or training data of DNN models, which is in many contexts not feasible or convenient. In this paper, we investigate black-box input diversity metrics as an alternative to white-box coverage criteria. To this end, we first select and adapt three diversity metrics and study, in a controlled manner, their capacity to measure actual diversity in input sets. We then analyse their statistical association with fault detection using two datasets and three DNN models. We further compare diversity with state-of-the-art white-box coverage criteria. Our experiments show that relying on the diversity of image features embedded in test input sets is a more reliable indicator than coverage criteria to effectively guide the testing of DNNs. Indeed, we found that one of our selected black-box diversity metrics far outperforms existing coverage criteria in terms of fault-revealing capability and computational time. Results also confirm the suspicions that state-of-the-art coverage metrics are not adequate to guide the construction of test input sets to detect as many faults as possible with natural inputs.

DeepGD: A Multi-Objective Black-Box Test Selection Approach for Deep Neural Networks

“…Similar to code coverage in traditional software systems [14], researchers have proposed several neuron coverage metrics in the context of DNN test selection to prioritize the selection of test inputs with high coverage scores [6][7][8][9][10]. Studies have shown the effectiveness of these approaches for distinguishing adversarial inputs but failed to demonstrate a positive correlation between coverage and DNN mispredictions [15][16][17][18]. Furthermore, for some coverage metrics, reaching maximum coverage can be easily achieved by selecting a few test inputs [15,19], thus putting into question the usefulness of such white-box DNN test selection approaches.…”

“…Moreover, the use of white-box DNN test selection approaches is hindered by the requirement for access to the internal structure of the DNN model or the training dataset. This can be a significant limitation, particularly when the model is proprietary or provided by a third party [16].…”

“…Another category of black-box metrics that has been successfully applied in testing traditional software systems is diversity [21][22][23]. However, only a few studies have investigated its effectiveness in testing DNN models [11,16,24]. Zhao et al [24] studied stateof-the-art (SOTA) test selection approaches for DNNs and found that they do not guarantee the selection of diverse test input sets.…”

“…Zhao et al [24] studied stateof-the-art (SOTA) test selection approaches for DNNs and found that they do not guarantee the selection of diverse test input sets. Also, Aghababaeyan et al [16] compared the fault-revealing power of several diversity metrics with coverage criteria. They found that geometric diversity (GD) [25] outperforms coverage criteria in terms of fault-revealing power and computational time.…”

“…Similar to failures in traditional software systems, many mispredictions result from the same faults in the DNN model and are therefore redundant [16,27]. This is why testing studies typically focus on faults, not failures [28,29].…”