Text Classification Using Label Names Only: A Language Model Self-Training Approach

Abstract

Current text classification methods typically require a good number of human-labeled documents as training data, which can be costly and difficult to obtain in real applications. Humans can perform classification without seeing any labeled examples but only based on a small set of words describing the categories to be classified. In this paper, we explore the potential of only using the label name of each class to train classification models on unlabeled data, without using any labeled documents. We use pre-trained neural language models both as general linguistic knowledge sources for category understanding and as representation learning models for document classification. Our method (1) associates semantically related words with the label names, (2) finds category-indicative words and trains the model to predict their implied categories, and (3) generalizes the model via self-training. We show that our model achieves around 90% accuracy on four benchmark datasets including topic and sentiment classification without using any labeled documents but learning from unlabeled data supervised by at most 3 words (1 in most cases) per class as the label name 1 .

X-Class: Text Classification with Extremely Weak Supervision

“…We also compare with LOTClass (Meng et al, 2020b), which works under the extremely weak supervision setting. In their experiments, it mostly relies on class names but has used a few keywords Table 1: An overview of our 7 benchmark datasets.…”

“…Compared with previous works Meng et al, 2020b), our X-Class has a significantly more mild requirement on human-provided class names in terms of quantity and quality. We have conducted an experiment in Table 4 for X-Class on 20News and NYT-Small by deleting all but one occurrence of a class name from the input corpus.…”

“…Interestingly, the performance of X-Class only drops less than 1%, still outperforming all compared methods. In contrast, the most recent work, LOTClass (Meng et al, 2020b), requires a wide variety of contexts of class names from the input corpus to ensure the quality of generated class vocabulary in its very first step.…”

“…Weakly supervised text classification. Weakly supervised text classification has attracted much attention from researchers (Tao et al, 2018;Meng et al, 2020a;Meng et al, 2020b). The general pipeline is to generate a set of document-class pairs to train a supervised model above them.…”

“…A recent work (Meng et al, 2020b) also studied the same topic -extremely weak supervision on text classification. It follows a similar idea of (Meng et al, 2020a) and further utilizes BERT to query replacements for class names to find keywords for classes, identifying potential classes for documents via string matching.…”