HTML text segmentation for Web page summarization by a key sentence extraction method

Abstract

SUMMARYThe information displayed as the search result by search engines is important for quickly finding the desired information. In particular, the summary of each Web page in the search results is important for determining the Web page content, as well as for determining how the input search term is used in each Web page, namely, the relation between the search term and the Web page. However, the summaries of the search results in conventional search engines have problems such as extracting only the opening text and not containing the search term, or containing the search term but having the sentence truncated in the middle so that the context of the term or the content of the Web page cannot be determined. Therefore, a summary in sentence units is desirable, but since HTML text includes many nonsentence items that do not contain punctuation, if they are unprocessed, it is difficult for a key sentence extraction system that treats sentences as units to provide a summary. Thus, in this paper, we propose an HTML text segmentation system that divides the source text of each Web page into meaningfully connected groups of text corresponding to sentences. We also verify experimentally that the text generated by this system can be used effectively in a Web page summarization.

Discussion Visualization on a Bulletin Board System

“…Word W is related to the topic if and only if the distance between W and one of the topic words is smaller than dmax. This threshold is defined by preliminary experimental results where a meaningful sentence includes at least seven words (Sunayama, Iyama, & Yachida, 2006). A related word should be included in three sentences: a sentence containing a topic word, the former, and the latter for the relationship.…”

Text Segmentation for Analysing Different Languages

“…For instance, studies of [5] [17], Arabic [11], Vietnamese [33], and Akkadian [35]. However, other studies [13] [29][22] applied sentence segmentation to analyse Japanese, Greek and Malay languages accordingly. As it is seen, there is a trend to apply text segmentation in the analysing text in different languages.…”