An Approach to Incremental Deep Web Crawling Based on Incremental Harvest Model

Huang, Qiuyan; Hong, Liang; Zhou, Yan

doi:10.1016/j.proeng.2012.01.093

Cited by 9 publications

(3 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…After randomly constructing the first query, subsequent queries use its result to build queries that are more efficient. Huang et al discussed incremental harvest model for incrementally crawling the deep Web. The model is constructed using records in the database at a different time interval from the local database.…”

Section: Current Status Of Web Crawlermentioning

confidence: 99%

A survey of Web crawlers for information retrieval

Kumar

Bhatia

Rattan

2017

WIREs Data Min & Knowl

View full text Add to dashboard Cite

Performance of any search engine relies heavily on its Web crawler. Web crawlers are the programs that get webpages from the Web by following hyperlinks. These webpages are indexed by a search engine and can be retrieved by a user query. In the area of Web crawling, we still lack an exhaustive study that covers all crawling techniques. This study follows the guidelines of systematic literature review and applies it to the field of Web crawling. We used the standard procedure of carrying out a systematic literature review on 248 studies from a total of 1488 articles published in 12 leading journals and other premier conferences and workshops. Existing literature about the Web crawler is classified into different key subareas. Each subarea is further divided according to the techniques being used. We analyzed the distribution of various articles using multiple criteria and depicted conclusions. Various studies that use open source Web crawlers are also reported. We have highlighted future areas of research. We call for an increased awareness in various fields of the Web crawler and identify how techniques from other domains can be used for crawling the Web. Limitations and recommendations for future are also discussed. WIREs Data Mining Knowl Discov 2017, 7:e1218. doi: 10.1002/widm.1218 This article is categorized under: Algorithmic Development > Web Mining Fundamental Concepts of Data and Knowledge > Information Repositories Fundamental Concepts of Data and Knowledge > Motivation and Emergence of Data Mining

show abstract

Section: Current Status Of Web Crawlermentioning

confidence: 99%

A survey of Web crawlers for information retrieval

Kumar

Bhatia

Rattan

2017

WIREs Data Min & Knowl

View full text Add to dashboard Cite

show abstract

“…Huanga, in at al [3]proposed an effective and efficient method is proposed to resolve this difficult. In the method, a set covering model is used to designate the web database based on this model an incremental harvest model is learned by the machine learning technique to choice the suitable query automatically.…”

Section: Related and Comparative Workmentioning

confidence: 99%

“…The difficulties with the conservative search engines can be not clever to index hidden web, deficiency of personalization, consequences with low precision and recall and not able to comprise user feedbacks founded on specific domain [3]. Alternative significant difficult is to discovery out and classify the access points of the hidden web databases i.e.…”

Section: Introductionmentioning

confidence: 99%

Knowledge base Construction using Hidden Web Retrieval Technique

Patel¹,

Ganatar²

2015

IJCA

View full text Add to dashboard Cite

Relations of algorithms for hidden web-focused information retrieval develop with it. When the stage the information retrieval, a huge difficulty arises and that is the information that one hidden web page can enclose manifold areas with extremely dissimilar information content. Hence the page has to be split into measurement and these parts examined separately for the results to be more precise. Web page segmentation and correlated technique have Therefore this paper suggests a combined approach that creates use of structural features and the visual features. It build a visual DOM tree on which the data records are recognized based on their structural similarity .The structure of these data records are reserved so that personage data items can be group effortlessly and precisely based on their visual features Which hidden web source do we intend at the information indispensable to access the data at the back web form and the type of interface. We proposed algorithm narrative vision based page segmentation (NVIPS) and also comparison DOM tree, VIPS.

show abstract

Formal concept analysis approach for data extraction from a limited deep web database

Zhang

Wei

2013

J Intell Inf Syst

View full text Add to dashboard Cite

An Approach to Incremental Deep Web Crawling Based on Incremental Harvest Model

Cited by 9 publications

References 7 publications

A survey of Web crawlers for information retrieval

A survey of Web crawlers for information retrieval

Knowledge base Construction using Hidden Web Retrieval Technique

Formal concept analysis approach for data extraction from a limited deep web database

Contact Info

Product

Resources

About