Web-page classification through summarization

Abstract

Web-page classification is much more difficult than pure-text classification due to a large variety of noisy information embedded in Web pages. In this paper, we propose a new Webpage classification algorithm based on Web summarization for improving the accuracy. We first give empirical evidence that ideal Web-page summaries generated by human editors can indeed improve the performance of Web-page classification algorithms. We then propose a new Web summarization-based classification algorithm and evaluate it along with several other state-of-the-art text summarization algorithms on the LookSmart Web directory. Experimental results show that our proposed summarization-based classification algorithm achieves an approximately 8.8% improvement as compared to pure-text-based classification algorithm. We further introduce an ensemble classifier using the improved summarization algorithm and show that it achieves about 12.9% improvement over pure-text based methods.

Image Description Mining and Hierarchical Clustering on Data Records Using HR-Tree

“…It is pointed out that document summarization will improve web page classification [5]. It has some relationship with our work since we both consider the selection of "important" words.…”

“…Traditionally, image search [1] and clustering [3] used image content to analyze its semantics. Some systems [4,5] based on CBIR were designed and implemented. There exists the problem that it is very hard to learn the semantic meaning of an image from low level visual features [2].…”

“…In traditional Content-Based Image Retrieval (CBIR) approaches [1,4,5], the similarity between two images is measured using color, brightness and texture etc. Other approaches [7 9 10] exploited textual information surrounding the images, which take the entire context as the image representation.…”

Identification and Classification of Web Pages with Specified Domain

“…In this with the help of HTML information seen in web page they are classifying the webpage with ANN classification technique. Dou Shen [4], Zheng Chen [4], Qiang Yang [4], Hua-JunZeng [4], Benyu Zhang [4], Yuchang Lu [4], Wei-Ying Ma [4] proposed mechanism of summarization which is used to categorize the web page. With the help of Web summarization algorithm, they have carried out the analysis of page-layout for the draw out of main topic of web page to increase the classification accuracy.…”

On Having Search Engines Deliver Hierarchies of Web Pages.

“…The methodology puts together four separate known techniques to solve an important unaddressed area of Web search. Relevant existing techniques include Web page classification algorithms [Brin and Page 1998, Chakrabarti 2002, Dumais and Chen 2002, Shen et al 2004, Shih and Karger 2004, Sun et al 2000 and machine learning algorithms [Nigam et al 2000, Raskutti et al 2002. The ultimate objective of our methodology is to minimize the need for people to resort to tedious manual navigations of semantically related Web pages in similar domains.…”