A Soft Computing Prefetcher to Mitigate Cache Degradation by Web Robots

Xie, Ning; Brown, Kyle; Rude, Nathan

doi:10.1007/978-3-319-59072-1_63

Cited by 3 publications

(1 citation statement)

References 13 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Отчеты компаний, которые занимаются мониторингом Интернета, показывают, что до 50 % трафика на сайте приходит от веб-робо-тов [3]. Различие статистических параметров поведения пользователей веб-ресурсов и веброботов можно использовать также для улучшения системы кэширования и настройки систем управления статистикой для исключения веб-роботов из различных маркетинговых отчетов [4].…”

unclassified

Метод Обнаружения Веб-Роботов На Основе Анализа Графа Пользовательского Поведения

Менщиков¹,

Menshchikov²,

Гатчин³

et al. 2019

ППС

View full text Add to dashboard Cite

unclassified

Метод Обнаружения Веб-Роботов На Основе Анализа Графа Пользовательского Поведения

Менщиков¹,

Menshchikov²,

Гатчин³

et al. 2019

ППС

View full text Add to dashboard Cite

Some (Non-)universal features of Web robot traffic

Zabihimayvan

2018

2018 52nd Annual Conference on Information Sciences and Systems (CISS)

View full text Add to dashboard Cite

Semantic Approach for Web-Robot Detection

Меньщиков¹,

Gatchin²

2019

vkit

View full text Add to dashboard Cite

Today modern researches suggest that robotic traffic on web resources prevails over user traffic in terms of volume and intensity. Web robots threaten data privacy, copyright, as well as affect performance, security, and affect statistics. There is a need to develop efficient detection and protection methods against web robots. Existing techniques involve the use of syntactic and analytical processing of web server logs to detect web robots. This article proposes to analyze the graph of visits of web robots, taking into account the time, as well as the connectivity of topics of the visited pages. In the article we provide an algorithm for data selection and cleansing, extracting semantic features of pages on a web resource, as well as the proposed detection parameters. We describe in detail the process of forming the ground truth and the principles of existing sessions labelling to the legit and robotic types. It is proposed to use the capabilities of a web server to identify sessions uniquely. The clustering procedure and the selection of a suitable classification model are discussed. For each of the studied models, the selection of hyper parameters and cross-validation of the results are made. The analysis of performance and detection accuracy, as well as comparison with the results of existing approaches is provided. Empirical results of the proposed method on web-resources show that this method leads to better web robot detection accuracy and precision comparing with the existing approaches.

show abstract

A Soft Computing Prefetcher to Mitigate Cache Degradation by Web Robots

Cited by 3 publications

References 13 publications

Метод Обнаружения Веб-Роботов На Основе Анализа Графа Пользовательского Поведения

Метод Обнаружения Веб-Роботов На Основе Анализа Графа Пользовательского Поведения

Some (Non-)universal features of Web robot traffic

Semantic Approach for Web-Robot Detection

Contact Info

Product

Resources

About