Clustering Web pages based on their structure

Crescenzi, Valter; Merialdo, Paolo; Missier, Paolo

doi:10.1016/j.datak.2004.11.004

Cited by 61 publications

(48 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…All these sources provide basic qualitative data, and are often used for reference or annotation in more specialized domains. We use a simple random sample of input values for search forms of these sites in orde r based on httpUnit 9 w lues.…”

Section: Results and Evaluationmentioning

confidence: 99%

“…2. We observe the concept of link-collection [9], which refers to anchor links in a page (class) that share the same path in the DOM tree, from the root element to their parent or grandparent element. As a result, these hyperlinks appear grouped together in the rendered page.…”

Section: Site-wide Wrapper Inductionmentioning

confidence: 99%

“…We borrow this terminology from [9], which describes a class of pages in a site as a collection of pages which are generated by the same server-side script or program. Different inputs to this script result in different instance pages.…”

Section: Page-level Wrapper Inductionmentioning

confidence: 99%

“…We are only aware of one approach to automatic site-structure discovery [9], which also constitutes the main motivation for our approach. The focus of their work is slightly different from ours.…”

Section: Related Workmentioning

confidence: 99%

See 3 more Smart Citations

Site-Wide Wrapper Induction for Life Science Deep Web Databases

Mir

Staab

Rojas

2009

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. We present a novel approach to automatic information extraction from Deep Web Life Science databases using wrapper induction. Traditional wrapper induction techniques focus on learning wrappers based on examples from one class of Web pages, i.e. from Web pages that are all similar in structure and content. Thereby, traditional wrapper induction targets the understanding of Web pages generated from a database using the same generation template as observed in the example set. However, Life Science Web sites typically contain structurally diverse web pages from multiple classes making the problem more challenging. Furthermore, we observed that such Life Science Web sites do not just provide mere data, but they also tend to provide schema information in terms of data labels -giving further cues for solving the Web site wrapping task. Our solution to this novel challenge of Site-Wide wrapper induction consists of a sequence of steps: 1. classification of similar Web pages into classes, 2. discovery of these classes and 3. wrapper induction for each class. Our approach thus allows us to perform unsupervised information retrieval from across an entire Web site. We test our algorithm against three real-world biochemical deep Web sources and report our preliminary results, which are very promising.

show abstract

Section: Results and Evaluationmentioning

confidence: 99%

Section: Site-wide Wrapper Inductionmentioning

confidence: 99%

Section: Page-level Wrapper Inductionmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Site-Wide Wrapper Induction for Life Science Deep Web Databases

Mir

Staab

Rojas

2009

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…Based on pattern vectors, similarity between pages is defined. To automatically extract main classes of pages offered by a website, [4] compares structures of DOM trees. In order to improve search results via text contents [1] uses the path length and [7] uses weighted path between two pages to adjust clusters.…”

Section: Related Workmentioning

confidence: 99%

Hierarchical Web-Page Clustering via In-Page and Cross-Page Link Structures

Lin

Han

et al. 2010

Advances in Knowledge Discovery and Data Mining

View full text Add to dashboard Cite

Abstract. Despite of the wide diversity of web-pages, web-pages residing in a particular organization, in most cases, are organized with semantically hierarchic structures. For example, the website of a computer science department contains pages about its people, courses and research, among which pages of people are categorized into faculty, staff and students, and pages of research diversify into different areas. Uncovering such hierarchic structures could supply users a convenient way of comprehensive navigation and accelerate other web mining tasks. In this study, we extract a similarity matrix among pages via in-page and crosspage link structures, based on which a density-based clustering algorithm is developed, which hierarchically groups densely linked webpages into semantic clusters. Our experiments show that this method is efficient and effective, and sheds light on mining and exploring web structures.

show abstract

Automatic Extraction of Logical Web Lists

Lanotte

Fumarola

Ceci

et al. 2014

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Recently, there has been increased interest in the extraction of structured data from the web (both "Surface" Web and"Hidden" Web). In particular, in this paper we focus on the automatic extraction of Web Lists. Although this task has been studied extensively, existing approaches are based on the assumption that lists are wholly contained in a Web page.They do not consider that many websites span their listing on several Web Pages and show for each of these only a partial view. Similar to databases, where a view can represent a subset of the data contained in a table, they split a logical list in multiple views (view lists). Automatic extraction of logical lists is an open problem. To tackle this issue we propose an unsupervised and domain-independent algorithm for logical list extraction. Experimental results on real-life and data-intensive Web sites confirm the effectiveness of our approach.

show abstract

Clustering Web pages based on their structure

Cited by 61 publications

References 24 publications

Site-Wide Wrapper Induction for Life Science Deep Web Databases

Site-Wide Wrapper Induction for Life Science Deep Web Databases

Hierarchical Web-Page Clustering via In-Page and Cross-Page Link Structures

Automatic Extraction of Logical Web Lists

Contact Info

Product

Resources

About