Extractive Based Single Document Text Summarization Using Clustering Approach

Bhole, Pankaj; Agrawal, Avinash J.

doi:10.11591/ijai.v3.i2.pp73-78

Section: Pre-processingmentioning

confidence: 99%

Optimizing Text Categorization for Indonesian Text Using Clustering Label Technique

et.al¹

2021

TURCOMAT

1

0

View full text Add to dashboard Cite

Text Categorization plays an important role for clustering the rapidly growing, yet unstructured, Indonesian text in digital format. Furthermore, it is deemed even more important since access to digital format text has become more necessary and widespread. There are many clustering algorithms used for text categorization. Unfortunately, clustering algorithms for text categorization cannot easily cluster the texts due to imperfect process of stemming and stopword of Indonesian language. This paper presents an intelligent system that categorizes Indonesian text documents into meaningful cluster labels. Label Induction Grouping Algorithm (LINGO) and Bisecting K- means are applied to process it through five phases, namely the pre-processing, frequent phrase extraction, cluster label induction, content discovery and final cluster formation. The experimental result showed that the system could categorize Indonesian text and reach to 93%. Furthermore, clustering quality evaluation indicates that text categorization using LINGO has high Precision and Recall with a value of 0.85 and 1, respectively, compare to Bisecting K-means which has a value of 0.78 and 0.99. Therefore, the result shows that LINGO is suitable for categorizing Indonesian text. The main contribution of this study is to optimize the clustering results by applying and maximizing text processing using Indonesian stemmer and stopword.

show abstract

“…The more important cluster will be in the top of ordered cluster list. The importance of a cluster can be determined based on the number of important or frequent words in it [14]. Each word in the cluster is tested based on the value of threshold θ.…”

Section: Cluster Ordering Phasementioning

confidence: 99%

“…The local sentence distribution weight of sentence is obtained by summing all of local sentence distribution weight of sentence's term and dividing it by the number of terms forming the sentence s i | |, as shown in Equation (14).…”

Section: Local Sentence Distribution With Pos Labelmentioning

confidence: 99%

Sentence Extraction Based on Sentence Distribution and Part of Speech Tagging for Multi-Document Summarization

Arifin

¹

,

Abdullah

²

,

Rosyadi

³

et al. 2018

View full text Add to dashboard Cite

Automatic multi-document summarization needs to find representative sentences not only b y sentence distrib ution to select the most important sentence b ut also b y how informative a term is in a sentence. Sentence distrib ution is suitab le for ob taining important sentences b y determining frequent and well-spread words in the corpus b ut ignores the grammatical information that indicates instructive content. The presence or ab sence of informative content in a sentence can b e indicated b y grammatical information which is carried b y part of speech (POS) lab els. In this paper, we propose a new sentence weighting method b y incorporating sentence distrib ution and POS tagging for multi -document summarization. Similarity-based Histogram Clustering (SHC) is used to cluster sentences in the data set. Cluster ordering is b ased on cluster importance to determine the important clusters. Sentence extraction b ased on sentence distrib ution and POS tagging is introduced to extract the representative sentences from the ordered clusters. The results of the experiment on the Document Unde rstanding Conferences (DUC) 2004 are compared with those of the Sentence Distrib ution Method. Our proposed method achieved b etter results with an increasing rate of 5.41% on ROUGE-1 and 0.62% on ROUGE-2.

show abstract

“…The local sentence distribution weight of sentence obtained by summing all of local sentence distribution weight of sentence's term and dividing it by the number of terms forming the sentence s i | |, as shown in Equation (14).…”

Section: Local Sentence Distribution With Pos Labelmentioning

confidence: 99%

Sentence Extraction Based on Sentence Distribution and Part of Speech Tagging for Multi-Document Summarization

Arifin¹,

Abdullah²,

Rosyadi³

et al. 2018

View full text Add to dashboard Cite

Automatic multi-document summarization needs to find representative sentences not only b y sentence distrib ution to select the most important sentence b ut also b y how informative a term is in a sentence. Sentence distrib ution is suitab le for ob taining important sentences b y determining frequent and well-spread words in the corpus b ut ignores the grammatical information that indicates instructive content. The presence or ab sence of informative content in a sentence can b e indicated b y grammatical information which is carried b y part of speech (POS) lab els. In this paper, we propose a new sentence weighting method b y incorporating sentence distrib ution and POS tagging for multi -document summarization. Similarity-based Histogram Clustering (SHC) is used to cluster sentences in the data set. Cluster ordering is b ased on cluster importance to determine the important clusters. Sentence extraction b ased on sentence distrib ution and POS tagging is introduced to extract the representative sentences from the ordered clusters. The results of the experiment on the Document Unde rstanding Conferences (DUC) 2004 are compared with those of the Sentence Distrib ution Method. Our proposed method achieved b etter results with an increasing rate of 5.41% on ROUGE-1 and 0.62% on ROUGE-2.

show abstract

Extractive Based Single Document Text Summarization Using Clustering Approach

Cited by 4 publications

References 9 publications

Optimizing Text Categorization for Indonesian Text Using Clustering Label Technique

Optimizing Text Categorization for Indonesian Text Using Clustering Label Technique

Sentence Extraction Based on Sentence Distribution and Part of Speech Tagging for Multi-Document Summarization

Sentence Extraction Based on Sentence Distribution and Part of Speech Tagging for Multi-Document Summarization

Contact Info

Product

Resources

About