Topic extraction using local graph centrality and semantic similarity

Abstract

Topic extraction is a challenging task under Natural Language Processing and Text Mining. Topic extraction is useful in natural language processing tasks such as automated summarization, question answering, and personalized search. In this paper, we propose an unsupervised topic extraction method using semantic similarity, keyword significance, and graph centrality. First, we select semantically similar words from text documents. Next, we perform disambiguation to find the correct senses of selected words. Then, we build a weighted graph using semantic relationships and significance of words in the text. Finally, we identify topic keywords using a novel concurrent local weighted centrality (LWWC) from words represented as nodes in a graph. Using standard annotated CiteULike and standard Brown datasets, we evaluated the results with precision, recall, and F-measure. We show that the proposed method yields results comparable with the state of the art LDA (Latent Dirichlet Allocation) and S-LDA (Sparse-LDA) topic extraction techniques. We also show that the proposed concurrent LWWC algorithm is more effective than the existing generic centrality measures in networks of words. We verified the statistical significance of improved effectiveness of our approach, using one-way analysis of variance (ANOVA) and Tukey-Honest Significant Difference (Tukey-HSD) post-hoc methods.