Deep Learning methods for Subject Text Classification of Articles

Abstract

Abstract-This work presents a method of classification of text documents using deep neural network with LSTM (long shortterm memory) units. We have tested different approaches to build feature vectors, which represent documents to be classified: we used feature vectors constructed as sequences of words included in the documents, or, alternatively, we first converted words into vector representations using word2vec tool and used sequences of these vector representations as features of documents. We evaluated feasibility of this approach for the task of subject classification of documents using a collection of Wikipedia articles representing 7 subject categories. Our experiments show that the approach based on an LSTM network with documents represented as sequences of words coded into word2vec vectors outperformed a standard, bag-of-word approach with documents represented as frequency-of-words feature vectors.

Application of Natural Language Processing Algorithms to the Task of Automatic Classification of Russian Scientific Texts

“…Lately, semantic model word2vec, based on neural network technologies, has been used. A number of recent studies have demonstrated the advantage of word2vec in comparison with previously used statistical approaches (for example, when it used in tandem with LSTM networks (Semberecki and Maciejewski, 2017)), although in another recent study (Wang et al 2017), the authors failed to demonstrate experimentally significant advantage of the semantic approach (as compared to statistical one) in experiments on the classification of texts with different number of class labels. Despite this, technology word2vec is considered to be a promising area for research, being actively developed over the past few years.…”

“…Methods of text preprocessing, considered in (Goncalves and Quaresma, 2018;Semberecki and Maciejewski, 2017), showed their effectiveness in case when it is necessary to train the model for classification of the English text. Nevertheless, replacement of stemming to lemmatization when working with the Russian language can significantly improves the quality of classification, since it is much easier to conduct POS-tagging for lemmatization of Russian words than for lemmatization of English words.…”

“…Based on works (Abuhaiba and Dawoud, 2017;Bourgonje et al 2018;Liu et al 2017;Semberecki and Maciejewski, 2017), it is possible to identify the models of machine learning that are most suitable for classification of textual data. Such models are: logistic regression, random forest, SVM, and artificial neural network (both feedforward and LSTM).…”

A Study into Math Document Classification using Deep Learning

“…In terms of deep neural networks [8], [9] and [10] studied different deep neural network models on the text classification task. Semberecki and Maciejewski applied long short-term memory (LSTM) model in documents classification to study different representations approaches [8].…”

“…In terms of deep neural networks [8], [9] and [10] studied different deep neural network models on the text classification task. Semberecki and Maciejewski applied long short-term memory (LSTM) model in documents classification to study different representations approaches [8]. Their evaluation showed that the vector representation approach outperformed a standard bag-ofword approach based on the LSTM model in the document classification task.In [9] a recurrent Convolutional Neural Network (CNN) model was proposed for text classification.…”