Formal Verification of Random Forests in Safety-Critical Applications

Törnblom, John; Nadjm‐Tehrani, Simin

doi:10.1007/978-3-030-12988-0_4

Cited by 15 publications

(18 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In our previous work [25], we verify safety-critical properties of random forests. Two techniques are presented, a fast but approximate technique which yields conservative output bounds, and a slower but precise technique employed when approximations are too conservative.…”

Section: Formal Verification Of Decision Trees and Tree Ensemblesmentioning

confidence: 99%

“…Their structural simplicity makes them easy to analyze systematically, but large (yet simple) models may still prove hard to verify due to combinatorial explosion. This paper is an improved and substantially extended version of our previous work [25] where we developed a method to partition the input domain of decision trees into disjoint sets, and to explore all path combinations in random forests in such a way that counteracts combinatorial path explosions. We implemented our method in a tool named VoRF, and evaluated the method on two case studies found in current literature.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Formal verification of input-output mappings of tree ensembles

Törnblom

Nadjm‐Tehrani

2020

Science of Computer Programming

Self Cite

View full text Add to dashboard Cite

Recent advances in machine learning and artificial intelligence are now being considered in safety-critical autonomous systems where software defects may cause severe harm to humans and the environment. Design organizations in these domains are currently unable to provide convincing arguments that their systems are safe to operate when machine learning algorithms are used to implement their software.In this paper, we present an efficient method to extract equivalence classes from decision trees and tree ensembles, and to formally verify that their input-output mappings comply with requirements. The idea is that, given that safety requirements can be traced to desirable properties on system input-output patterns, we can use positive verification outcomes in safety arguments. This paper presents the implementation of the method in the tool VoTE (Verifier of Tree Ensembles), and evaluates its scalability on two case studies presented in current literature. We demonstrate that our method is practical for tree ensembles trained on low-dimensional data with up to 25 decision trees and tree depths of up to 20. Our work also studies the limitations of the method with high-dimensional data and preliminarily investigates the trade-off between large number of trees and time taken for verification.

show abstract

Section: Formal Verification Of Decision Trees and Tree Ensemblesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Formal verification of input-output mappings of tree ensembles

Törnblom

Nadjm‐Tehrani

2020

Science of Computer Programming

Self Cite

View full text Add to dashboard Cite

show abstract

“…Related Work. It is only recently that adversarial attacks and robustness of tree ensemble classifiers started to be a subject of investigation (Andriushchenko and Hein 2019;Chen et al 2019a;Einziger et al 2019;Kantchelian, Tygar, and Joseph 2016;Törnblom and Nadjm-Tehrani 2018;Törnblom and Nadjm-Tehrani 2019b;Törnblom and Nadjm-Tehrani 2019a). The most related work is the robustness verification tool of tree ensembles by Törnblom and Nadjm-Tehrani (2019b) called VoTE.…”

Section: Introductionmentioning

confidence: 99%

Abstract Interpretation of Decision Tree Ensemble Classifiers

Ranzato

Zanella

2020

AAAI

View full text Add to dashboard Cite

We study the problem of formally and automatically verifying robustness properties of decision tree ensemble classifiers such as random forests and gradient boosted decision tree models. A recent stream of works showed how abstract interpretation, which is ubiquitously used in static program analysis, can be successfully deployed to formally verify (deep) neural networks. In this work we push forward this line of research by designing a general and principled abstract interpretation-based framework for the formal verification of robustness and stability properties of decision tree ensemble models. Our abstract interpretation-based method may induce complete robustness checks of standard adversarial perturbations and output concrete adversarial attacks. We implemented our abstract verification technique in a tool called silva, which leverages an abstract domain of not necessarily closed real hyperrectangles and is instantiated to verify random forests and gradient boosted decision trees. Our experimental evaluation on the MNIST dataset shows that silva provides a precise and efficient tool which advances the current state of the art in tree ensembles verification.

show abstract

“…Second, such an explanation can also be used as an approximation of the original model. For example, verification of machine learning models is another popular topic, but a general sound and complete verification algorithm for ensemble trees have proven impractical [26]. As a step back, we can look at the software testing scenario: since the explanation mimics the behaviour of the original model, if it violates a property, then it is likely that the original model would fail the verification, too.…”

Section: Introductionmentioning

confidence: 99%

Extracting Optimal Explanations for Ensemble Trees via Logical Reasoning

Zhang¹,

Hóu²,

Huang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Ensemble trees are a popular machine learning model which often yields high prediction performance when analysing structured data. Although individual small decision trees are deemed explainable by nature, an ensemble of large trees is often difficult to understand. In this work, we propose an approach called optimised explanation (OptExplain) that faithfully extracts global explanations of ensemble trees using a combination of logical reasoning, sampling and optimisation. Building on top of this, we propose a method called the profile of equivalent classes (ProClass), which uses MAX-SAT to simplify the explanation even further 1 . Our experimental study on several datasets shows that our approach can provide high-quality explanations to large ensemble trees models, and it betters recent top-performers.

show abstract

Formal Verification of Random Forests in Safety-Critical Applications

Cited by 15 publications

References 12 publications

Formal verification of input-output mappings of tree ensembles

Formal verification of input-output mappings of tree ensembles

Abstract Interpretation of Decision Tree Ensemble Classifiers

Extracting Optimal Explanations for Ensemble Trees via Logical Reasoning

Contact Info

Product

Resources

About