A Method for Focused Crawling Using Combination of Link Structure and Content Similarity

Jamali, Mohsen; Sayyadi, Hassan; Hariri, Babak Bagheri; Abolhassani, Hassan

doi:10.1109/wi.2006.19

Cited by 29 publications

(19 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Jamali, H. Sayyadi, B. B. Hariri and H. Abolhassani [12] uses a combination of the link structure of web documents and content similarity of web documents with respect to the given topic. They treated hyperlinks and contents present in the web pages as the authors view about other web pages and it relates them to a particular domain.…”

Section: Related Workmentioning

confidence: 99%

Improved Architecture of Focused Crawler on the basis of Content and Link Analysis

Singh¹,

Gupta²,

Singh³

2017

IJMECS

View full text Add to dashboard Cite

show abstract

Section: Related Workmentioning

confidence: 99%

Improved Architecture of Focused Crawler on the basis of Content and Link Analysis

Singh¹,

Gupta²,

Singh³

2017

IJMECS

View full text Add to dashboard Cite

show abstract

“…The Best-First [4] algorithm focuses on the retrieval of pages which are relevant to a particular given topic. It's an algorithm that uses a score to define which page has a best score.…”

Section: The Best-first Searchmentioning

confidence: 99%

Focused Web Crawling Algorithms

Amrin¹,

Xia²,

Dai³

2015

JCP

View full text Add to dashboard Cite

Nowadays the web is rich of any kind of information. And this information is freely available thanks to the hypermedia information systems and the Internet. This information greatly influenced our lives, our lifestyle and way of thinking. A web search engine is a complex multi-level system that helps us to search the information that available on the Internet. A web crawler is one of the most important parts of the search engine. It's a robot that systematically browses and indexes the World Wide Web. A focused web crawler is used crawling only web pages that are relevant to the user given topic or web page link. A focused crawler is a part of the search system that helps user to find most relevant information from the Internet. In our days, this area of computer science is very popular and important for the development of science and technology, because The Internet is constantly and rapidly growing, and the information in it. In this article we will review most effective focused web crawling algorithms that determines the fate of the search system.

show abstract

“…Others are based on meta search and content block partition, as in [18]. Jamali et al [12] used a combination of link structure of the fetched pages and the content similarity of a document to a certain domain. This mixture of link structure and content is used to compute a ranking score for the candidate unfetched pages.…”

Section: Related Workmentioning

confidence: 99%

An Agent-Based Focused Crawling Framework for Topic- and Genre-Related Web Document Discovery

Pappas

Katsimpras

Stamatatos

2012

2012 IEEE 24th International Conference on Tools With Artificial Intelligence

View full text Add to dashboard Cite

Abstract-The discovery of web documents about certain topics is an important task for web-based applications including web document retrieval, opinion mining and knowledge extraction. In this paper, we propose an agent-based focused crawling framework able to retrieve topic-and genre-related web documents. Starting from a simple topic query, a set of focused crawler agents explore in parallel topic-specific web paths using dynamic seed URLs that belong to certain web genres and are collected from web search engines. The agents make use of an internal mechanism that weighs topic and genre relevance scores of unvisited web pages. They are able to adapt to the properties of a given topic by modifying their internal knowledge during search, handle ambiguous queries, ignore irrelevant pages with respect to the topic and retrieve collaboratively topic-relevant web pages. We performed an experimental study to evaluate the behavior of the agents for a variety of topic queries demonstrating the benefits and the capabilities of our framework.

show abstract

A Method for Focused Crawling Using Combination of Link Structure and Content Similarity

Cited by 29 publications

References 6 publications

Improved Architecture of Focused Crawler on the basis of Content and Link Analysis

Improved Architecture of Focused Crawler on the basis of Content and Link Analysis

Focused Web Crawling Algorithms

An Agent-Based Focused Crawling Framework for Topic- and Genre-Related Web Document Discovery

Contact Info

Product

Resources

About