Template Extraction from Heterogeneous Web Pages with Cosine Similarity

Abstract

Now a day's detection of templates from a large number of web pages has received a lot of attention. Template detection technique improves the performance of clustering, classification & search engines. In our work we proposed a novel algorithm by using cosine similarity based Template Extraction. We are using the cosine similarity approach to cluster the web documents. With the help of underlying structure of web documents we found the template for individual cluster. Our experimental evaluation show that our approach is effective in terms of computing Time and Clustering cost.

Information Retrieval Based on DOM Trees

“…Regarding the template detection techniques, we found in the literature, some of them used artificial benchmarks [25], while others used real heterogeneous web pages [114,10]. Similarly, some authors selected the input web pages randomly [120,116,50], while others provided the input web pages manually [114,113]. Finally, regarding the block detection techniques, we found authors that used well-known benchmark suites such as CleanEval [20] benchmark suite [115,99], MSS [85], L3S-GN1 [61], etc.…”

“…Using different collections of benchmarks to compare template detectors or content extractors is highly inaccurate because some techniques used artificial benchmarks [25] (automatically generated web pages that share exactly the same template) while others used real heterogenous web pages implemented by different designers [114,10,6]. In the same way, some authors selected the web pages randomly [120,116,50] possibly implementing different templates, while others manually provided web pages that implement exactly the same template [114,113]. Finally, other authors used well-known benchmark suites such as CleanEval [20] benchmark suite [115,99], MSS (Myriad 40 and Big 5) [85], L3S-GN1 [61], etc.…”