Explaining Vulnerabilities to Adversarial Machine Learning through Visual Analytics

Abstract

1 2 Recall: 0.81 in the poisoned model 0.90 in the victim model 7 5 4 B A C E D F G 3 G.1 6 G.2 Fig. 1. Reliability attack on spam filters.(1) Poisoning instance #40 has the largest impact on the recall value, which is (2) also depicted in the model overview.(3) There is heavy overlap among instances in the two classes as well the poisoning instances. (4) Instance #40 has been successfully attacked causing a number of innocent instances to have their labels flipped. (5) The flipped instances are very close to the decision boundary. (6) On the feature of words "will" and "email", the variances of poisoning instances are large. (7) A sub-optimal target (instance #80) has less impact on the recall value, but the cost of insertions is 40% lower than that of instance #40.Abstract-Machine learning models are currently being deployed in a variety of real-world applications where model predictions are used to make decisions about healthcare, bank loans, and numerous other critical tasks. As the deployment of artificial intelligence technologies becomes ubiquitous, it is unsurprising that adversaries have begun developing methods to manipulate machine learning models to their advantage. While the visual analytics community has developed methods for opening the black box of machine learning models, little work has focused on helping the user understand their model vulnerabilities in the context of adversarial attacks. In this paper, we present a visual analytics framework for explaining and exploring model vulnerabilities to adversarial attacks. Our framework employs a multi-faceted visualization scheme designed to support the analysis of data poisoning attacks from the perspective of models, data instances, features, and local structures. We demonstrate our framework through two case studies on binary classifiers and illustrate model vulnerabilities with respect to varying attack strategies.

Boxer: Interactive Comparison of Classifier Results

“…Black box methods are often designed for very specific goals. Ma et al [MXLM20], for example, support the identification of feature combinations that elicit a specific response from a model. Ye et al [YXX*19] enable users to assess and increase the quality of training data labels.…”

A survey of visual analytics techniques for machine learning

“…Based on these measures, developers can iteratively identify those features that cause discrimination and remove them from the model. Researchers are also interested in exploring potential vulnerabilities in models that prevent them from being reliably applied to real-world applications [84,91]. Cao et al [84] proposed AEVis to analyze how adversarial examples fool neural networks.…”

“…It then employs a river-based metaphor to show the diverging and merging patterns of the extracted datapaths, which reveal where the adversarial samples mislead the model. Ma et al [91] designed a series of visual representations from overview to detail to reveal how data poisoning will make a model misclassify a specific sample. By comparing the distributions of the poisoned and normal training data, experts can deduce the reason for the misclassification of the attacked sample.…”

The State of the Art in Enhancing Trust in Machine Learning Models with the Use of Visualizations

“…Security vulnerabilities. When research is conducted in ML, there is always a factor that is not often taken into account at first: “how do we secure ML models from unethical attacks?” An instance of this idea is published by Ma et al [MXLM20], explaining how visualization can assist in avoiding vulnerabilities of adversarial attacks in ML. Specifically, their focus is on how to avoid data poisoning attacks from the models, data instances, features, and local structures perspectives with the use of their VA approach.…”

“…Biologists and doctors, for instance, are interested in being able to compare data structures and receive guidance on where to focus on. Ma et al [MXLM20] employ a multi‐faceted visualization schema intended to aid the analysis of ML experts for the domain of adversarial attacks.…”