Email Shape Analysis

Abstract

Abstract. Email has become an integral part of everyday life. Without a second thought we receive bills, bank statements, and sales promotions all to our inbox. Each email has hidden features that can be extracted. In this paper, we present a new mechanism to characterize an email without using content or context called Email Shape Analysis. We explore the applications of the email shape by carrying out a case study; botnet detection and two possible applications: spam filtering, and social-context based finger printing. Our in-depth analysis of botnet detection leads to very high accuracy of tracing templates and spam campaigns. However, when it comes to spam filtering we do not propose new method but rather a complementing method to the already high accuracy Bayesian spam filter. We also look at its ability to classify individual senders in personal email inbox's.

Spam filtering by quantitative profiles

“…Motivated by Sroufe et al [12], we have proposed the quantitative profile approach to email classification. In this report we explored two quantitative profiles, the line profile and the character profile.…”

“…Sroufe et al [12] suggest to filter spam by means of its shape, which the authors define (using our terminology) as a smoothed line profile of email body, where smoothing is performed by the kernel smoother. Sroufe et al also report the total error of 30%, based on a preliminary study on the TREC corpus.…”

“…Further, the authors find the performance very good, 'considering that no content or context was even referenced', cf. [12], p. 26. The line profile, that is inspired by the email shape analysis, attains on the TREC corpus fpr = 4.23% a fnr = 17.00%, when the threshold is not optimized and email headers are intentionally not taken into account, cf.…”

“…Instead of the "bag-of-words" representation, employed in text-mining and naive Bayes filtering, in the quantitative profile (QP ) approach that we propose, an email is represented by an m-dimensional vector of numbers with m fixed in advance. Inspired by Sroufe, Phithakkitnukoon, Dantu, and Cangussu [12], two instances of QP s are considered: line profile (LP ) and character profile (CP ). Informally put, the line profile of an email is a vector of lengths of the first m lines.…”