Formal verification of input-output mappings of tree ensembles

Abstract

Recent advances in machine learning and artificial intelligence are now being considered in safety-critical autonomous systems where software defects may cause severe harm to humans and the environment. Design organizations in these domains are currently unable to provide convincing arguments that their systems are safe to operate when machine learning algorithms are used to implement their software.In this paper, we present an efficient method to extract equivalence classes from decision trees and tree ensembles, and to formally verify that their input-output mappings comply with requirements. The idea is that, given that safety requirements can be traced to desirable properties on system input-output patterns, we can use positive verification outcomes in safety arguments. This paper presents the implementation of the method in the tool VoTE (Verifier of Tree Ensembles), and evaluates its scalability on two case studies presented in current literature. We demonstrate that our method is practical for tree ensembles trained on low-dimensional data with up to 25 decision trees and tree depths of up to 20. Our work also studies the limitations of the method with high-dimensional data and preliminarily investigates the trade-off between large number of trees and time taken for verification.

Abstract Interpretation of Decision Tree Ensemble Classifiers

“…It is worth observing that the worst case verification time of silva never exceeds 1 minute and that the average verification time on the hardest input samples is always less than 5 seconds. (Törnblom and Nadjm-Tehrani 2019b), already discussed in Section 1. We replicated the experiments on the MNIST dataset as described in (Törnblom and Nadjm-Tehrani 2019b) and compared the results in terms of robustness and verification time (on our machine).…”

“…(Törnblom and Nadjm-Tehrani 2019b), already discussed in Section 1. We replicated the experiments on the MNIST dataset as described in (Törnblom and Nadjm-Tehrani 2019b) and compared the results in terms of robustness and verification time (on our machine). Each experiment is run on RFs and GBDTs trained with the same parameters: RFs are trained using scikit-learn with Gini/average parameters, while GBDTs are trained by CatBoost with default learning rate and softmax voting scheme.…”

“…We used silva for inferring stability, robustness, fragility, vulnerability and breakage (as defined in Section 3) of random forest classifiers on MNIST, which have been trained with different combinations of number of trees, maximum tree depth, splitting criteria (Gini and entropy impurity measures) and voting schemes (average and max). We compared the performance of silva against the robustness verification tool called VoTE (Törnblom and Nadjm-Tehrani 2019b), already discussed in Section 1, both for random forests and gradient boosted decision trees. In our experiments RFs have been trained by scikit-learn while CatBoost has been used for GBDTs.…”

“…Our experimental evaluation on the standard MNIST dataset and on the Sensorless dataset used by (Chen et al 2019a) shows that silva provides a precise and efficient tool both for deriving the robustness metrics of forest classifiers and for performing a complete check of the robustness to adversarial perturbations of input samples. Our evaluation also compared the performance of silva against a recent robustness verification tool of tree ensembles (Törnblom and Nadjm-Tehrani 2019b), and this showed that silva improves on the current state of the art. The following three images display: on the left, an original image O from the test set of MNIST, correctly classified as 7 by a random forest using 100 decision trees with maximum depth 100; in the middle, an image A which is automatically generated by silva as adversarial attack of O for a perturbation ±1 of the brightness values of its pixels; on the right, a gray/black/white image showing which pixels change from O to A where gray means unchanged, black +1, white −1.…”

“…It is only recently that adversarial attacks and robustness of tree ensemble classifiers started to be a subject of investigation (Andriushchenko and Hein 2019;Chen et al 2019a;Einziger et al 2019;Kantchelian, Tygar, and Joseph 2016;Törnblom and Nadjm-Tehrani 2018;Törnblom and Nadjm-Tehrani 2019b;Törnblom and Nadjm-Tehrani 2019a). The most related work is the robustness verification tool of tree ensembles by Törnblom and Nadjm-Tehrani (2019b) called VoTE. This is an abstraction-refinement procedure which iteratively refines a partition of the input space where each block of the partition is an hyperrectangle.…”