Text classification using document-document semantic similarity

Abstract

Abstract:One of the key problems encountered while using a text classification learning algorithms is that they require huge amount of labelled examples to learn accurately. The objective of this paper is to propose a novel method of topic modelling and document-document semantic similarity algorithm (DDSSA), which reduces the need for larger training data. This algorithm finds the concepts and keywords of the unlabelled text, identifying the topic of unlabelled text from list of concepts and keywords obtained from labelled text. This can be achieved by obtaining the concepts of the labelled text and identify the keywords which holds strong relationships with given labelled data. This topics and keywords obtained from the labelled text can be stored in the database which in turn can be used to compute the semantic similarity with concepts obtained from the unlabelled text. The proposed method is compared with the popular latent semantic analysis (LSA) applied in NLTK and Mallet datasets. The experiment result shows that the proposed method is superior to LSA in most of the cases.