Filtering spam from bad neighborhoods

Abstract

SUMMARYOne of the most annoying problems on the Internet is spam. To fight spam, many approaches have been proposed over the years. Most of these approaches involve scanning the entire contents of e-mail messages in an attempt to detect suspicious keywords and patterns. Although such approaches are relatively effective, they also show some disadvantages. Therefore an interesting question is whether it would be possible to effectively detect spam without analyzing the entire contents of e-mail messages. The contribution of this paper is to present an alternative spam detection approach, which relies solely on analyzing the origin (IP address) of e-mail messages, as well as possible links within the e-mail messages to websites (URIs). Compared to analyzing suspicious keywords and patterns, detection and analysis of URIs is relatively simple. The IP addresses and URIs are compared to various kinds of blacklists; a hit increases the probability of the message being spam. Although the idea of using blacklists is well known, the novel idea proposed within this paper is to introduce the concept of 'bad neighborhoods'. To validate our approach, a prototype has been developed and tested on our university's mail server. The outcome was compared to SpamAssassin and mail server log files. The result of that comparison was that our prototype showed remarkably good detection capabilities (comparable to SpamAssassin), but puts only a small load on the mail server.

Internet Bad Neighborhoods temporal behavior

“…Bad Neighborhoods can be seen as subnetworks having higher concentrations of malicious IP addresses than the measured average [1], [5]- [8]. For example, we found in a previous study that 50% of spamming IP addresses on the Internet can be traced back to only 20 Autonomous Systems (ASes) [8].…”

“…Based on those works, the Bad Neighborhood concept was introduced in [1]. The authors developed a mail filter that employed Spam BadHoods to judge whether a message was spam.…”

“…The Internet Bad Neighborhood concept emerged [1] from the observation that malicious IP addresses are not evenly distributed over the IP addressing space [2]- [4]. Bad Neighborhoods can be seen as subnetworks having higher concentrations of malicious IP addresses than the measured average [1], [5]- [8].…”

“…The bad neighborhood concept furthers traditional blacklisting and improves attack prediction by extending the reputation of malicious IP addresses to their immediate neighbors -i.e., by blacklisting their neighboring IP addresses, which are, in turn, more likely to carry out attacks due to the typical concentration. As shown in recent works [1], [7], this idea has proved to be effective in predicting and protecting network from attacks.…”

“…In [1], van Wanrooij and Pras developed a bad neighborhood-based spam filter, while in [5] we have put spamming bad neighborhoods under scrutiny. The issue of the bad neighborhood size was investigated in [6], while the performance of third-party spam blacklists was evaluated in [7].…”