Improving web spam classifiers using link structure

Abstract

Web spam has been recognized as one of the top challenges in the search engine industry [14]. A lot of recent work has addressed the problem of detecting or demoting web spam, including both content spam [16,12] and link spam [22,13]. However, any time an anti-spam technique is developed, spammers will design new spamming techniques to confuse search engine ranking methods and spam detection mechanisms. Machine learning-based classification methods can quickly adapt to newly developed spam techniques. We describe a two-stage approach to improve the performance of common classifiers. We first implement a classifier to catch a large portion of spam in our data. Then we design several heuristics to decide if a node should be relabeled based on the preclassified result and knowledge about the neighborhood. Our experimental results show visible improvements with respect to precision and recall.

Graph regularization methods for Web spam detection

“…With spam, hyperlink direction is of great importance since we do not expect genuine hosts to link to spam hosts even when links in the opposite direction are quite common. This has been empirically confirmed in Castillo et al (2007), Gan and Suel (2007). Thus, we will also consider "positive distance squared" as a distortion measure, that is where…”

“…Another option is to extract link-based metrics for each node and use these as features in any standard classification algorithm . Finally, it has been shown that the link-based information can be used to refine the results of a base classifier by re-labelling using propagation through the hyperlink graph, or a stacked classifier (Castillo et al 2007;Gan and Suel 2007).…”

On the robustness of google scholar against spam

“…They can create dummy web sites which link to the website they want to push (link farms), exchange links with other webmasters, buy links on third party web pages, and post links to their websites, for instance, in blogs. To detect link spam, much research has been performed, among others [5][6][7][8][9][10][11][12].…”

Improving web spam detection with re-extracted features

“…[5] implemented a classifier to catch a large portion of spam, then several heuristics rules were designed to decide whether a node should be relabeled. [2] summarized the existing content and link based method, detected web spam with machine learning algorithms, then gave some heuristic rules to improve the performance.…”

“…Both [2] and [5] achieved good results with the preliminary machine learning algorithms, but they optimized the detection result with some heuristic rules. As we all know, effective spam detection is essentially an "arms race" between search engines and spamers.…”