A Hierarchical Neural-Network-Based Document Representation Approach for Text Classification

Abstract

Document representation is widely used in practical application, for example, sentiment classification, text retrieval, and text classification. Previous work is mainly based on the statistics and the neural networks, which suffer from data sparsity and model interpretability, respectively. In this paper, we propose a general framework for document representation with a hierarchical architecture. In particular, we incorporate the hierarchical architecture into three traditional neural-network models for document representation, resulting in three hierarchical neural representation models for document classification, that is, TextHFT, TextHRNN, and TextHCNN. Our comprehensive experimental results on two public datasets, that is, Yelp 2016 and Amazon Reviews (Electronics), show that our proposals with hierarchical architecture outperform the corresponding neural-network models for document classification, resulting in a significant improvement ranging from 4.65% to 35.08% in terms of accuracy with a comparable (or substantially less) expense of time consumption. In addition, we find that the long documents benefit more from the hierarchical architecture than the short ones as the improvement in terms of accuracy on long documents is greater than that on short documents.

Hybrid Distributed Techniques for Lower-dimensional Representation of Documents in the Scientific Literature

“…The new model has been tested against the original Doc2Vec and several other methods on some standard classification and sentiment analysis tasks. Other approaches in [76,77,78] have also proposed novel ANN-based structures to project document text into a lower dimensional vector.…”

Publication of Court Records: Circumventing the Privacy-Transparency Trade-Off

“…prediction-based or count-based methods-to real-valued vectors along with the context in which they are used [42]. Multiple variations of this structure exist [29,34,36,37,64]. This approach has been used for example in [39] to rank and explain influential aspects of law, or by [44] to predict the most relevant sources of law for any given piece of text using "neural networks and deep learning algorithms".…”

Sentiment classification of social media reviews using an ensemble classifier

“…Chandrasekaran because of their superior classification ability [21]. The authors of the paper "A Hierarchical Neural-Network Based Document Representation Approach for Text Classification" [22] integrate hierarchical neural architecture into traditional neural network methods and showed that their proposals outperform the corresponding neural network models for document classification. Nurulhuda and Ali [23] have mentioned three different weighting schemes to generate the word vectors which are Term Frequency-Inverse Document Frequency Binary Occurrence and Term Occurrence.…”