Botnet Sizes: When Maths Meet Myths

Chiapponi, Elisa; Daciér, Marc; Todisco, Massimiliano; Catakoglu, Onur; Thonnard, Olivier

doi:10.1007/978-3-030-76352-7_52

Cited by 1 publication

(1 citation statement)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This service facilitates the rotation of proxies, overcoming rate limitations on APIs or websites that restrict the number of requests a web crawler can make within a specified timeframe. By utilizing a pool of proxies, we strategically distribute requests across different IP addresses, making it challenging for targeted web-sites to effectively block our crawling operations [11].…”

Section: Methodsmentioning

confidence: 99%

Evaluating and Improving the Robustness of Web Crawlers Against IP Blocking and Captchas

Onyenagubom

2024

IJLTEMAS

View full text Add to dashboard Cite

In this work, we demonstrate the capability of a JavaScript-based web crawler to overcome anti-crawling measures such as CAPTCHAs and IP blocking. we delved into the ethical and legal dimensions of web crawling and provide recommendations for future research endeavors in this domain. A web crawler, being an automated software program, can navigate through websites and extract information. While it serves purposes like website analysis and indexing, it can also be misused for extracting personal data, scraping content, and overloading servers. Website administrators often employ anti-crawling techniques, such as CAPTCHAs, Robot.txt, and IP blocking, to thwart malicious web crawlers from accessing their content. These techniques aim to curtail the ability of a web crawler to scrape, access, or overload website resources, without impeding legitimate users from accessing the necessary content. The objective of this study is to demonstrate and enhance the resilience of legitimate web crawlers against anti-crawling techniques like CAPTCHAs and IP blocking, challenging the notion that these measures are universally effective against all types of web crawlers.

show abstract

Section: Methodsmentioning

confidence: 99%

Evaluating and Improving the Robustness of Web Crawlers Against IP Blocking and Captchas

Onyenagubom

2024

IJLTEMAS

View full text Add to dashboard Cite

show abstract

Botnet Sizes: When Maths Meet Myths

Cited by 1 publication

References 10 publications

Evaluating and Improving the Robustness of Web Crawlers Against IP Blocking and Captchas

Evaluating and Improving the Robustness of Web Crawlers Against IP Blocking and Captchas

Contact Info

Product

Resources

About