Self-Organizing Maps in the Design of Anti-spam Filters - A Proposal based on Thematic Categories

Cabrera-León, Ylermi; Báez, Patricio García; Suárez-Araujo, Carmen Paz

doi:10.5220/0006041400210032

Cited by 6 publications

(4 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Unsupervised Self-Organized Map based systems have also been explored by researchers such as Cabrera-León et al [15]. Their introduced system works on 13 different categories for emails.…”

Section: Relevent Studiesmentioning

confidence: 99%

“…The Rand Index (RI) determines a similarity score between two sets of clustering by considering each of the pairs of the provided samples and summing up pairs that are assigned in identical or different clusters in the true clustering in addition to the predicted ones [53]. The raw RI score is then 'adjusted for chance' into the ARI using (15) The index as depicted in Fig. 20, clearly projects the superior quality of clusters produced by OPTICS and DBSCAN, whereas the rest, according to ARI, did not show any commendable outcome.…”

Section: E2a Adjusted Rand Index (Ari)mentioning

confidence: 99%

See 1 more Smart Citation

An Unsupervised Approach for Content-Based Clustering of Emails Into Spam and Ham Through Multiangular Feature Formulation

Karim¹,

Azam

Shanmugam

et al. 2021

IEEE Access

View full text Add to dashboard Cite

The rapid growth of spam email attacks and the inherent malicious dynamism within those attacks on a range of social, personal and business activities warrants an intelligent and automated anti-spam framework. Attempts like malware propagation, identity theft, sensitive data pilfering, monetary as well as reputational damage are sharply increasing, endangering the privacy of the victim. Current solutions that are rather incomplete when the multidimensional feature range of email, is taken into account. We believe a methodology based on Artificial Intelligence, especially unsupervised machine learning is the way forward. This research attempts to investigating the application of unsupervised learning for the clustering of Spam and Ham emails. The overall goal of the research is to develop an unsupervised framework that solely depends on unsupervised methodologies through a clustering approach that includes multiple algorithms, primarily using the email content (body) and the subject header. The clustering has been done on a novel binary dataset of 22,000 entries of ham and spam emails, composed of ten features (reduced from eleven to ten after the feature reduction). Seven out of these ten features are unique to this study, engineered to represent impactful analytical email characteristics from a multiangular point of view. Out of five different clustering algorithms investigated in this work, OPTICS produced the optimum clustering demonstrating a 0.26% higher average efficacy than its nearest performer DBSCAN. The average balanced accuracy for OPTICS and DBSCAN was found to be ≈75.76%.

show abstract

“…Unsupervised Self-Organized Map based systems have also been explored by researchers such as Cabrera-León et al [15]. Their introduced system works on 13 different categories for emails.…”

Section: Relevent Studiesmentioning

confidence: 99%

Section: E2a Adjusted Rand Index (Ari)mentioning

confidence: 99%

An Unsupervised Approach for Content-Based Clustering of Emails Into Spam and Ham Through Multiangular Feature Formulation

Karim¹,

Azam

Shanmugam

et al. 2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…(2016) [74] Discussed a Self-Organizing-Map (SOM) based system where emails were categorized in 13 different categories using raw Term Frequency measure along with some other metadata.…”

Section: 27mentioning

confidence: 99%

Efficient Clustering of Emails Into Spam and Ham: The Foundational Study of a Comprehensive Unsupervised Framework

et al. 2020

View full text Add to dashboard Cite

The spread and adoption of spam emails in malicious activities like information and identity theft, malware propagation, monetary and reputational damage etc. are on the rise with increased effectiveness and diversification. Without doubt these criminal acts endanger the privacy of many users and businesses'. Several research initiatives have taken place to address the issue with no complete solution until now; and we believe an intelligent and automated methodology should be the way forward to tackle the challenges. However, till date limited studies have been conducted on the applications of purely unsupervised frameworks and algorithms in tackling the problem. To explore and investigate the possibilities, we intend to propose an anti-spam framework that fully relies on unsupervised methodologies through a multi-algorithm clustering approach. This paper presents an in-depth analysis on the methodologies of the first component of the framework, examining only the domain and header related information found in email headers. A novel method of feature reduction using an ensemble of 'unsupervised' feature selection algorithms has also been investigated in this study. In addition, a comprehensive novel dataset of 100,000 records of ham and spam emails has been developed and used as the data source. Key findings are summarized as follows: I) out of six different clustering algorithms used -Spectral and K-means demonstrated acceptable performance while OPTICS projected the optimum clustering with an average of 3.5% better efficiency than Spectral and Kmeans, validated through a range of validations processes II) The other three algorithms-BIRCH, HDBSCAN and K-modes, did not fare well enough. III) The average balanced accuracy for the optimum three algorithms has been found to be ≈94.91%, and IV) The proposed feature reduction framework achieved its goal with high confidence.

show abstract

“…Cabrera-León et al [207] introduced another SOM based system where they used 13 different categories for the emails. The researchers had started by a 4-stage preprocessing of emails (both spam and ham).…”

Section: B: Self-organizing Map Based Propositionsmentioning

confidence: 99%

A Comprehensive Survey for Intelligent Spam Email Detection

et al. 2019

View full text Add to dashboard Cite

The tremendously growing problem of phishing e-mail, also known as spam including spear phishing or spam borne malware, has demanded a need for reliable intelligent anti-spam e-mail filters. This survey paper describes a focused literature survey of Artificial Intelligence (AI) and Machine Learning (ML) methods for intelligent spam email detection, which we believe can help in developing appropriate countermeasures. In this paper, we considered 4 parts in the email's structure that can be used for intelligent analysis: (A) Headers Provide Routing Information, contain mail transfer agents (MTA) that provide information like email and IP address of each sender and recipient of where the email originated and what stopovers, and final destination. (B) The SMTP Envelope, containing mail exchangers' identification, originating source and destination domains\users. (C) First part of SMTP Data, containing information like from, to, date, subject-appearing in most email clients (D) Second part of SMTP Data, containing email body including text content, and attachment. Based on the number the relevance of an emerging intelligent method, papers representing each method were identified, read, and summarized. Insightful findings, challenges and research problems are disclosed in this paper. This comprehensive survey paves the way for future research endeavors addressing theoretical and empirical aspects related to intelligent spam email detection. INDEX TERMS Machine learning, phishing attack, spear phishing, spam detection, spam email, spam filtering.

show abstract

Self-Organizing Maps in the Design of Anti-spam Filters - A Proposal based on Thematic Categories

Cited by 6 publications

References 0 publications

An Unsupervised Approach for Content-Based Clustering of Emails Into Spam and Ham Through Multiangular Feature Formulation

An Unsupervised Approach for Content-Based Clustering of Emails Into Spam and Ham Through Multiangular Feature Formulation

Efficient Clustering of Emails Into Spam and Ham: The Foundational Study of a Comprehensive Unsupervised Framework

A Comprehensive Survey for Intelligent Spam Email Detection

Contact Info

Product

Resources

About