Efficient Training of Robust Decision Trees Against Adversarial Examples

Robust Optimal Classification Trees against Adversarial Examples

“…Then to compare the practical performance we run the algorithms on eight popular datasets (Chen et al 2019;Vos and Verwer 2020) and varying perturbation radii ( ). All of our experiments ran on 15 Intel Xeon CPU cores and 72 GB of RAM total, where each algorithm ran on a single core.…”

“…This flexibility comes at a cost in run-time, as it uses an iterative solver to optimize this score for each split it learns. GROOT (Vos and Verwer 2020) improve the greedy procedure by efficiently computing the worst-case Gini impurity and allowing users to specify boxshaped attacker perturbation limits. In this paper, we compare against GROOT and TREANT as these greedy methods achieve state-of-the-art scores.…”

“…In previous works (Calzavara et al 2020;Vos and Verwer 2020) attacker capabilities were arbitrarily chosen but this limits the value of algorithm comparisons, shown in Figure 5. In this figure we vary the L ∞ radius by which an adversary can perturb samples.…”

“…While breakthroughs in machine learning research have enabled training of powerful predictive models, most models are still vulnerable to adversarial examples, samples with tiny perturbations that cause them to be misclassified. Since the discovery of adversarial examples in neural networks (Szegedy et al 2013) much work has gone into training models that are robust to these attacks and recently, the first efforts were made to train robust decision trees against adversarial examples (Chen et al 2019;Calzavara et al 2020;Vos and Verwer 2020). However, the current methods are greedy and offer no performance guarantees.…”

“…This model is robust in the sense that it predicts the correct ground-truth label in a box of specified size surrounding each sample, this optimizes robustness against corrupted instances (Diochnos, Mahloujifar, and Mahmoody 2018). Like existing robust decision tree learning algorithms (Calzavara et al 2020;Vos and Verwer 2020), ROCT allows users to specify a boxshaped attack model that encodes an attacker's capability to modify feature values with the aim of maximizing loss. Existing robust decision tree learning methods use a greedy node splitting approach.…”