Adapted Web Crawler for Mining Offline Web Data using AFHC

Abstract

Adaptive focused hyperlink crawler (AFHC) aim to search the entire inner level sub link of the web pages related to a specific topic and to download unique web pages to local disk. The download web page information searched in offline browsing and avoids the repeated searches in the web server to give a solution to problem. The major problem is to retrieve the maximal set of relevant and quality pages. Crawler software can be retrieve web pages by hyperlinks through internet. The number of web sites using web crawlers cannot retrieve the relevant pages. The system have browser and search engine. Browser using AFHC reduce time to finding accurate content in web pages in the hyperlinks and also restrict to download web pages to local disk. Search engine using extended cocitaion algorithm to retrieve accurate content in the local disk and search based on any word, all word and phrase matching in the local disk. It is useful to the student and organization. The research is easily found to personalize the crawl history and search history for knowing the user transactions.

Web Scraping as a scientific tool for theoretical reference

“…It allows the retrieval of elements using criteria such as tags, classes, identi ers, attributes, and even the content of the text. This capability enables the selective extraction of data, saving time and effort 42 . Beautiful Soup provides methods for navigating the tree of elements, allowing access to related elements in a hierarchical manner.…”