Building Robust Phishing Detection System: an Empirical Analysis

Lee, Je-Hyun; Ye, Pingxiao; Liu, Ruofan; Divakaran, Dinil Mon; Chan, Mun Choon

doi:10.14722/madweb.2020.23007

Cited by 15 publications

(11 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In future work we will explore the degree to which the defense outlined by Lee et al is effective against FIGA [27]. We also will explore other applications of FIGA to test its effectiveness in domains such as android malware or network intrusion detection.…”

Section: Discussionmentioning

confidence: 99%

“…Lee et al is the prior work that most closely aligns to our approach. They propose a defense against an adversarial attack on phishing detection models [27].…”

Section: Related Workmentioning

confidence: 99%

“…Lee et al proposed a defense against an attack similar to FIGA, which replaces the values of most important features of the phishing site with the corresponding values from a benign site [27]. The attack designed by Lee et al showed that the AUPRC (area under the precision recall curve) score of their phishing detection random forest model dropped from 99.1 to 69.4.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Feature Importance Guided Attack: A Model Agnostic Adversarial Attack

Gressel¹,

Hegde²,

Sreekumar³

et al. 2021

Preprint

View full text Add to dashboard Cite

Machine learning models are susceptible to adversarial attacks which dramatically reduce their performance. Reliable defenses to these attacks are an unsolved challenge [11]. In this work we present a novel evasion attack: the 'Feature Importance Guided Attack' (FIGA) which generates adversarial evasion samples. FIGA is model agnostic, it assumes no prior knowledge of the defending model's learning algorithm, but does assume knowledge of the feature representation. FIGA leverages feature importance rankings; it perturbs the most important features of the input in the direction of the target class we wish to mimic. We demonstrate FIGA against eight phishing detection models. We keep the attack realistic by perturbing phishing website features that an adversary would have control over. Using FIGA we are able to cause a a reduction in the F1-score of a phishing detection model from 0.96 to 0.41 on average. Finally, we implement adversarial training as a defense against FIGA and show that while it is sometimes effective, it can be evaded by changing the parameters of FIGA.

show abstract

Section: Discussionmentioning

confidence: 99%

“…Lee et al is the prior work that most closely aligns to our approach. They propose a defense against an adversarial attack on phishing detection models [27].…”

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Feature Importance Guided Attack: A Model Agnostic Adversarial Attack

Gressel¹,

Hegde²,

Sreekumar³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…We consider the phishing webpage dataset [2] which is used in [41] for phishing detection with supervised learning. While the phishing URLs were obtained by crawling PhishTank feed [7], the benign pages were crawled randomly from the top 300,000 websites as ranked by Alexa [3].…”

Section: Phishing Webpage Detection Datasetmentioning

confidence: 99%

“…While there has been a rapid increase in machine learning (ML) applications, they often require accurately labeled datasets to achieve competitive performance. This is also common in the security domain where supervised classifiers are built for threat detection (e.g., for detecting phishing attacks [9,[39][40][41][42][43], malware and malicious communications [10-12, 47, 48, 51, 55]). However, even with significant and costly human verification [1,45], these datasets are prone to errors and poisoning attacks.…”

Section: Introductionmentioning

confidence: 99%

Markov Chain Monte Carlo-Based Machine Unlearning: Unlearning What Needs to be Forgotten

Nguyen,

Oikawa,

Divakaran

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

As the use of machine learning (ML) models is becoming increasingly popular in many real-world applications, there are practical challenges that need to be addressed for model maintenance. One such challenge is to 'undo' the effect of a specific subset of dataset used for training a model. This specific subset may contain malicious or adversarial data injected by an attacker, which affects the model performance. Another reason may be the need for a service provider to remove data pertaining to a specific user to respect the user's privacy. In both cases, the problem is to 'unlearn' a specific subset of the training data from a trained model without incurring the costly procedure of retraining the whole model from scratch. Towards this goal, this paper presents a Markov chain Monte Carlo-based machine unlearning (MCU) algorithm. MCU helps to effectively and efficiently unlearn a trained model from subsets of training dataset. Furthermore, we show that with MCU, we are able to explain the effect of a subset of a training dataset on the model prediction. Thus, MCU is useful for examining subsets of data to identify the adversarial data to be removed. Similarly, MCU can be used to erase the lineage of a user's personal data from trained ML models, thus upholding a user's "right to be forgotten". We empirically evaluate the performance of our proposed MCU algorithm on real-world phishing and diabetes datasets. Results show that MCU can achieve a desirable performance by efficiently removing the effect of a subset of training dataset and outperform an existing algorithm that utilizes the remaining dataset.

show abstract