Clustering of semantically enriched short texts

Abstract

The paper is devoted to the issue of clustering small sets of very short texts. Such texts are often incomplete and highly inconclusive, so establishing a notion of proximity between them is a challenging task. In order to cope with polysemy we adapt the SenseSearcher algorithm (SnS), by Kozlowski and Rybinski in Computational Intelligence 33(3): 335-367, 2017b. In addition, we test the possibilities of improving the quality of clustering ultra-short texts by means of enriching them semantically. We present two approaches, one based on neural-based distributional models, and the other based on external knowledge resources. The approaches are tested on SnSRC and other knowledge-poor algorithms. Keywords Document clustering • Information retrieval • Semantic enrichment Participants can react to the results and create a dialogue. All the participants' responses are free text answers. The answers are short, often incomplete, and highly biased toward the questions, so establishing a measure of proximity between such texts is a difficult issue. Although we begin with a specific task, we would like to analyze it in a more general way. Kozlowski and Rybinski (2017b) have shown interesting features of the SnS algorithm, which is a novel text mining approach to sense induction. SnS is a knowledge-poor algorithm, and to a large extent it is a language independent approach. It has been also shown that, based on SnS, it is possible to build an efficient text clustering algorithm, named SnSRC. Unfortunately it turns out that for applications that work on small repositories of very short texts the clustering quality of SnSRC is not as good as for typical applications with medium to large texts clustering. In this paper we investigate how SnSRC can be modified so that it can be efficiently used to cluster short texts. In particular, the goal of this paper is to revise our approach so that SnS can be used for clustering very short texts generated ad hoc in a brainstorming session; this will mean that so that the manual work of ordering the above mentioned discussion material is minimized. This paper is an extension of Kozlowski and Rybinski (2017a), presented on ISMIS '2017. Here, we provide a detailed description of the SnSRC algorithm, and present its improvements, as compared to the ISMIS version. Additionally, in this paper, we discuss various ways of expanding text features in more detail. We have verified a number of well-known short text clustering approaches, including data-centric, description-centric, and semantically-enhanced ones. Here, we provide more experiments with SnSRC and other clustering algorithms, combining the considered algorithms with data expansion methods, and showing how, in general, the semantic enrichment can influence the quality of clustering algorithms. The paper is structured as follows: In Section 2 we discuss related work. Section 3 is devoted to presenting general concepts concerning SnSRC. Then in Section 4 we present our approach in more detail. Section 5 discusses the experiments and Sectio...

Discovering Topic Representative Terms for Short Text Clustering

“…Recently, some artificial neural network based methods were developed for short text representation learning. Kozlowski and Rybinski [24] have used neural network based distributional semantic model for enriching the semantic meaning of short text for clustering. Similarly, Xu et al [4] have proposed STC 2 , which adopts deep learning techniques for short text representation learning.…”

ETM: Enrichment by topic modeling for automated clinical sentence classification to detect patients’ disease history

“…Impressive progress has been made on the problem of text classification, but few studies have tackled sentence classification (Kozlowski and Rybinski 2019;Zelikovitz and Hirsh 2000;Cheng et al 2014;Yin et al 2017;Khoo et al 2006;Kim 2014;Jurafsky and Martin 2019;Aggarwal 2018). Unlike the traditional text classification problem, sentence classification pose two main challenges.…”

“…Several techniques have been proposed to tackle the challenges posed by sentence classification, including dimension reduction (Zelikovitz and Hirsh 2000;Sriram et al 2010;Khoo et al 2006;Bollegala et al 2018), topic modeling (Cheng et al 2014;Chen et al 2011;Yang et al 2015), clustering (Kozlowski and Rybinski 2019;Yin et al 2017;Bollegala et al 2018;Dai et al 2013;Kozlowski and Rybinski 2017;Yang et al 2019), and word embedding (Kozlowski and Rybinski 2019;Kim 2014;Lee and Dernoncourt 2016;Hill et al 2016). Kim (Kim 2014) proposed a single layer of CNN applied for sentence classification.…”

“…Many researchers have examined the task of mining clinical text for applications in healthcare (Demner-Fushman et al 2009;Sevenster et al 2015;Friedman et al 2004;Byrd et al 2014;Torii et al 2015;Khalifa and Meystre 2015;Kozlowski and Rybinski 2019;Shen et al 2018) and have approached it as a basic text classification problem. Two major challenges in clinical text classification are the unstructured and short representations of text.…”

Text Mining in Healthcare: Bringing Structure to Electronic Health Records

“…Many researchers have examined the task of mining clinical text for applications in healthcare (Demner-Fushman et al, 2009;Sevenster et al, 2015;C. Friedman, Shagina, Lussier, & Hripcsak, 2004;Byrd et al, 2014;Torii et al, 2015;Khalifa & Meystre, 2015;Kozlowski & Rybinski, 2019;Shen et al, 2018) and have approached it as a basic text classification problem. Two major challenges in clinical 2.1.…”