Feature selection for Spam and Phishing detection

Abstract

Unsolicited Bulk Email (UBE) has become a large problem in recent years. The number of mass mailers in existence is increasing dramatically. Automatically detecting UBE has become a vital area of current research. Many email clients (such as Outlook and Thunderbird) already have junk filters built in. Mass mailers are continually evolving and overcoming some of the junk filters. This means that the need for research in the area is ongoing. Many existing techniques seem to randomly choose the features that will be used for classification. This paper aims to address this issue by investigating the utility of over 40 features that have been used in recent literature. Information gain for these features are calculated over Ham, Spam and Phishing corpora.

Using Feature Selection and Classification Scheme for Automating Phishing Email Detection

“…We used 4 phishing dataset files as presented in Table 3. These phishing datasets have been used in phishing detection research including work by [3], [4], [5], [6], [9], [10], [12], and [16]. In order to provide non-phishing datasets, we used the SpamAssassin Project [17] from the easy ham directory.…”

“…If not, we set the value 1 to indicate the email is abnormal. This feature has been proposed by [5].…”

“…To measure the effectiveness of our approach, we use four metrics that are also used in previous work [5], [6] …”

“…Several good anti-techniques such as content-based [6], [11], [16] and behavior-based [7], [5], [13] have been developed to address the phishing problems. However, phishing attacks have continued to be a serious problem.…”

Improvement of Email Threats Detection by User Training

“…Despite good anti-techniques such as content-based [5][6][7] and behavior-based [8][9][10][11] ones, the attacks are still a serious problem because phishes continually change their ways to defeat the anti-phishing techniques. Also, most existing emails filtering approaches are static easy to defeat by modifying emails content and link strings.…”