Can machine learning be secure?

Barreno, Marco; Nelson, Brent D.; Sears, Russell; Joseph, Anthony D.; Tygar, J. D.

doi:10.1145/1128817.1128824

Cited by 693 publications

(656 citation statements)

References 34 publications

Supporting

Mentioning

650

Contrasting

Unclassified

Order By: Relevance

“…In a previous paper, we categorize attacks against machine learning systems along three axes [1]. The axes of the taxonomy are: • Causative attacks influence learning with control over training data.…”

Section: Attacksmentioning

confidence: 99%

“…In training, SpamBayes computes a spam score vector P (S) where the i th component is a token spam score for the i th token given by ( , ) ( ) ( ) ( ) (1) where N S , N H , N S (i), and N H (i) are the number of spam emails, ham emails, spam emails including the i th token and ham emails including the i th token, respectively. The quantity P (S,i) is an estimate of Pr(E is spam | e i ) if the prior of ham and spam are equal, but for our purposes, it is simply a per-token score for the email.…”

Section: °= ®°Ementioning

confidence: 99%

“…We consider a single attack spam message, after which the counts become ( ) We wish to analyze the quantity (1) …”

Section: A2 Effect Of Poisoning On Token Scoresmentioning

confidence: 99%

“…However, in many real applications an attacker might have the ability to provide a machine learning system with maliciously chosen inputs that cause the system to infer poor classification rules. In the spam domain, for example, the adversary can send carefully crafted spam messages 1 Comp. Sci.…”

Section: Introductionmentioning

confidence: 99%

“…Unlike attacks that exploit flaws in an application's implementation or policies, our attacks take advantage of the learner's adaptive nature. Each attack embodies a particular insight about ways in which a machine learning system is vulnerable, which we can better understand in terms of several commonalities between the attacks.In a previous paper, we categorize attacks against machine learning systems along three axes [1]. The axes of the taxonomy are: • Causative attacks influence learning with control over training data.…”

mentioning

confidence: 99%

See 4 more Smart Citations

Misleading Learners: Co-opting Your Spam Filter

Nelson

Barreno

Jack

et al. 2009

Machine Learning in Cyber Trust

Self Cite

124

View full text Add to dashboard Cite

Using statistical machine learning for making security decisions introduces new vulnerabilities in large scale systems. We show how an adversary can exploit statistical machine learning, as used in the SpamBayes spam filter, to render it useless-even if the adversary's access is limited to only 1% of the spam training messages. We demonstrate three new attacks that successfully make the filter unusable, prevent victims from receiving specific email messages, and cause spam emails to arrive in the victim's inbox. IntroductionApplications use statistical machine learning to perform a growing number of critical tasks in virtually all areas of computing. The key strength of machine learning is adaptability; however, this can become a weakness when an adversary manipulates the learner's environment. With the continual growth of malicious activity and electronic crime, the increasingly broad adoption of learning makes assessing the vulnerability of learning systems to attack an essential problem.The question of robust decision making in systems that rely on machine learning is of interest in its own right. But for security practitioners, it is especially important, as a wide swath of security-sensitive applications build on machine learning technology, including intrusion detection systems, virus and worm detection systems, and spam filters [13,14,18,20,24].Past machine learning research has often proceeded under the assumption that learning systems are provided with training data drawn from a natural distribution of inputs. However, in many real applications an attacker might have the ability to provide a machine learning system with maliciously chosen inputs that cause the system to infer poor classification rules. In the spam domain, for example, the adversary can send carefully crafted spam messages 1 Comp. Sci. Div., Soda Hall #1776, University of California, Berkeley, 94720-1776, USA 17and Reliability, DOI: 10.1007/978-0-387-88735-7_2, In Machine Learning in Cyber Trust: Security, Privacy, Reliability, eds. J. Tsai and P..Yu (eds.) Springer, 2009, pp. 17-51 18 that a human user will correctly identify and mark as spam, but which can influence the underlying machine learning system and adversely affect its ability to correctly classify future messages.We demonstrate how attackers can exploit machine learning to subvert the SpamBayes statistical spam filter. Our attack strategies exhibit two key differences from previous work: traditional attacks modify attack instances to evade a filter, whereas our attacks interfere with the training process of the learning algorithm and modify the filter itself; and rather than focusing only on placing spam emails in the victim's inbox, we also present attacks that remove legitimate emails from the inbox.We consider attackers with one of two goals: expose the victim to an advertisement or prevent the victim from seeing a legitimate message. Potential revenue gain for a spammer drives the first goal, while the second goal is motivated, for example, by an organiza...

show abstract

Section: Attacksmentioning

confidence: 99%

Section: °= ®°Ementioning

confidence: 99%

“…We consider a single attack spam message, after which the counts become ( ) We wish to analyze the quantity (1) …”

Section: A2 Effect Of Poisoning On Token Scoresmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations

Misleading Learners: Co-opting Your Spam Filter

Nelson

Barreno

Jack

et al. 2009

Machine Learning in Cyber Trust

Self Cite

124

View full text Add to dashboard Cite

show abstract

Security Monitoring and Management in 5G

2019

Wiley 5G Ref

View full text Add to dashboard Cite

Monitoring is an essential functionality that is required to provide the awareness and automation necessary to manage both performance and security of fixed and mobile networks. The advent of 5G introduces new concepts that both facilitate the monitoring and security management but have also introduced new vulnerabilities and challenges that need to be addressed. This section presents the different requirements, vulnerabilities, and concepts related to security monitoring and management. It introduces some of the solutions that are being developed and tested to obtain efficient security in 5G networks. It introduces the different network security monitoring techniques, software‐defined security and network slicing, cognitive network security management, liability management, and techniques to obtain efficient security. It also introduces recent trends that profit from 5G network programmability to optimize the security monitoring and management.

show abstract