ScrAPIr

Alrashed, Tarfah; Almahmoud, Jumana; Zhang, Amy X.; Karger, David R.

doi:10.1145/3313831.3376691

Cited by 12 publications

(2 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Adhering strictly to website terms of service is paramount to avoid any legal complications. ScrapingBee stands out as our go-to solution, thanks to its impressive success rate, well-documented features, and an API that makes handling headless browsers a breeze [16]. Fig.…”

Section: Methodsmentioning

confidence: 99%

Evaluating and Improving the Robustness of Web Crawlers Against IP Blocking and Captchas

Onyenagubom

2024

IJLTEMAS

View full text Add to dashboard Cite

In this work, we demonstrate the capability of a JavaScript-based web crawler to overcome anti-crawling measures such as CAPTCHAs and IP blocking. we delved into the ethical and legal dimensions of web crawling and provide recommendations for future research endeavors in this domain. A web crawler, being an automated software program, can navigate through websites and extract information. While it serves purposes like website analysis and indexing, it can also be misused for extracting personal data, scraping content, and overloading servers. Website administrators often employ anti-crawling techniques, such as CAPTCHAs, Robot.txt, and IP blocking, to thwart malicious web crawlers from accessing their content. These techniques aim to curtail the ability of a web crawler to scrape, access, or overload website resources, without impeding legitimate users from accessing the necessary content. The objective of this study is to demonstrate and enhance the resilience of legitimate web crawlers against anti-crawling techniques like CAPTCHAs and IP blocking, challenging the notion that these measures are universally effective against all types of web crawlers.

show abstract

Section: Methodsmentioning

confidence: 99%

Evaluating and Improving the Robustness of Web Crawlers Against IP Blocking and Captchas

Onyenagubom

2024

IJLTEMAS

View full text Add to dashboard Cite

show abstract

“…To make the HTTP request, the get method is used via the URL, https://www.googleapis.com/youtube/v3/search, and it is necessary to fill in a literal parameter called PART, which specifies a comma-separated list of one or more search resource properties that will be included in the API response. The items that can be included in the part parameter value are ID and SNIPPET [20,21].…”

Section: B Video Extractionmentioning

confidence: 99%

Educational Video Recommender System

Timmi

2024

IJIET

View full text Add to dashboard Cite

In recent years, informal education has witnessed a significant upsurge, fueled by technological advancements and the ubiquitous availability of online educational content. Internet users, including students, researchers, and teachers, are increasingly seeking supplementary educational resources across diverse online repositories to augment their knowledge. Within this landscape, recommendation systems emerge as indispensable tools, aiding users in the discovery of pertinent resources aligned with their academic interests. This article proposes a novel recommendation methodology leveraging a hybrid approach, incorporating both Content-Based Filtering (CBF) and Collaborative Filtering (CF) algorithms. By harnessing information from a myriad of data repositories, this system excels in identifying and presenting the most relevant and desirable educational resources, with a particular focus on meeting the needs of students. This holistic approach embraces user profiles, contextual information, and supplementary data, underscoring its potential to revolutionize informal education in the digital age.

show abstract