From Feature Engineering and Topics Models to Enhanced Prediction Rates in Phishing Detection

Abstract

Phishing is a type of fraud attempt in which the attacker, usually by e-mail, pretends to be a trusted person or entity in order to obtain sensitive information from a target. Most recent phishing detection researches have focused on obtaining highly distinctive features from the metadata and text of these e-mails. The obtained attributes are then used to feed classification algorithms in order to determine whether they are phishing or legitimate messages. In this paper, it is proposed an approach based on machine learning to detect phishing e-mail attacks. The methods that compose this approach are performed through a feature engineering process based on natural language processing, lemmatization, topics modeling, improved learning techniques for resampling and cross-validation, and hyperparameters configuration. The first proposed method uses all the features obtained from the Document-Term Matrix (DTM) in the classification algorithms. The second one uses Latent Dirichlet Allocation (LDA) as an operation to deal with the problems of the ''curse of dimensionality'', the sparsity, and the text context portion included in the obtained representation. The proposed approach reached marks with an F1-measure of 99.95% success rate using the XGBoost algorithm. It outperforms state-of-the-art phishing detection researches for an accredited data set, in applications based only on the body of the e-mails, without using other e-mail features such as its header, IP information or number of links in the text.INDEX TERMS Feature engineering, feature extraction, natural language processing, phishing detection, topics modeling, XGBoost.

The Answer is in the Text: Multi-Stage Methods for Phishing Detection Based on Feature Engineering

“…Considering our previous proposal [47], the approach presented in this paper are results of an improvement over it, since it employs the new Stanza NLP toolkit as the database in the feature engineering process to promote tokenization, POS tagging, and lemmatization (instead of WordNet, as done in [47]); and proposes methods to select feature based on statistical measures, and methods to extract new features, from that initially presented in DTM, based on PCA and LSA (instead of LDA, as done in [47]).…”

“…This strategy for folding was not yet used for phishing detection approaches based on ML algorithms. In this context, our previous work presented in [47] and this approach were the first employment of this plan.…”

“…Gualberto et al [47] had the goal of obtaining highly distinctive features for phishing detection from the text of the e-mails. They proposed an approach based on machine learning performed through a feature engineering process based on natural language processing, lemmatization, topics modeling (using LDA), improved learning techniques for resampling and cross-validation, and hyperparameters configuration.…”

“…These two compilations of e-mails are both public datasets and their employment to evaluate phishing detection approaches is a widespread practice [45], [50]. The approaches presented in [47], [51], [45], [42], [31], [29], [33], [52], [24], [26], [27] and [25] are validated using the SpamAssassin Public Corpus and the Nazario Phishing Corpus as datasets sources.…”

“…Similarly, as explained in [69], B d,m maps terms to topics (b i,j is the weight of term i in the topic j) and Σ m,m C t,m = S maps topics to documents (s i,j is the weight of the topic i in the documents j). In this way, LSA would be used in a similar way as Latent Dirichlet Allocation (LDA) is used in [47].…”