Weakly-Supervised Hierarchical Text Classification

Abstract

Hierarchical text classification, which aims to classify text documents into a given hierarchy, is an important task in many real-world applications. Recently, deep neural models are gaining increasing popularity for text classification due to their expressive power and minimum requirement for feature engineering. However, applying deep neural networks for hierarchical text classification remains challenging, because they heavily rely on a large amount of training data and meanwhile cannot easily determine appropriate levels of documents in the hierarchical setting. In this paper, we propose a weakly-supervised neural method for hierarchical text classification. Our method does not require a large amount of training data but requires only easy-to-provide weak supervision signals such as a few class-related documents or keywords. Our method effectively leverages such weak supervision signals to generate pseudo documents for model pre-training, and then performs self-training on real unlabeled data to iteratively refine the model. During the training process, our model features a hierarchical neural structure, which mimics the given hierarchy and is capable of determining the proper levels for documents with a blocking mechanism. Experiments on three datasets from different domains demonstrate the efficacy of our method compared with a comprehensive set of baselines 1 .

HiGitClass: Keyword-Driven Hierarchical Classification of GitHub Repositories

“…c p (κ i ) is a normalization constant. Following [23], we choose the number of vMF components differently for leaf and internal categories: (1) For a leaf category C j , the number of components m is set to 1 and the mixture model degenerates to a single vMF distribution. (2) For an internal category C j , we set the number of components to be the number of C j 's children in the label hierarchy.…”

“…[26] proposes a graph-CNN based model to convert text to graph-of-words, on which the graph convolution operations are applied for feature extraction. Under weakly-supervised or dataless settings, previous approaches include HierDataless [33], WeSHClass [23] and PCNB/PCEM [41], which have been introduced in Section IV-A. All above mentioned studies focus on text data without additional information.…”

Weakly-Supervised Deep Recurrent Neural Networks for Basic Dance Step Generation

“…Recent findings demonstrate the remarkable performance of applying deep learning in motion generation [7]- [9], music processing [10], [11] and several other tasks. An advantage of using deep learning is the relatively low effort for feature engineering [12]. Deep learning also enables an end-to-end mapping between the input and output features, reducing the requirement for intermediate hand-crafted annotations [10].…”

“…Motivated by the benefits of deep learning, in this study, we explore the application of deep learning models in the generation of basic dance steps. Though deep learning offers the benefits mentioned above, applying it to dance generation can be challenging because of the following three main issues: 1) deep learning models may not generate variable-length non-linear sequences [13] such as dance; 2) the given model may not be able to constrain the motion beat [1], [2], [14] to music beat, and 3) the performance of deep learning models is proportional to the number of training datasets; thus, they require large carefully-labeled datasets for good performance [12].…”

“…To constrain the motion beat to the music beat [1], we assume that a music beat may be present when a motion beat appears. To avoid additional hand-crafted annotations and reduce the amount of training data, the contrastive cost function employs weak labels (see [12], [17], [18]) that are generated by motion direction. Weak labels may yield incomplete, inaccurate or incorrect data [19].…”