Online multi-label dependency topic models for text classification

Abstract

Multi-label text classification is an increasingly important field as large amounts of text data are available and extracting relevant information is important in many application contexts. Probabilistic generative models are the basis of a number of popular text mining methods such as Naive Bayes or Latent Dirichlet Allocation. However, Bayesian models for multi-label text classification often are overly complicated to account for label dependencies and skewed label frequencies while at the same time preventing overfitting. To solve this problem we employ the same technique that contributed to the success of deep learning in recent years: greedy layer-wise training. Applying this technique in the supervised setting prevents overfitting and leads to better classification accuracy. The intuition behind this approach is to learn the labels first and subsequently add a more abstract layer to represent dependencies among the labels. This allows using a relatively simple hierarchical topic model which can easily be adapted to the online setting. We show that our method successfully models dependencies online for large-scale multi-label datasets with many labels and improves over the baseline method not modeling dependencies. The same strategy, layer-wise greedy training, also makes the batch variant competitive with existing more complex multi-label topic models.

Partial Multi-Label Learning with Noisy Label Identification

“…There are many studies trying to exploit the label correlations for enhancing the multilabel learning (Zhu, Kwok, and Zhou 2017;Huang, Yu, and Zhou 2012). Some of them focus on pairwise correlation (Li, Song, and Luo 2017), while some others consider high order correlation among all labels (Burkhardt and Kramer 2018;Read et al 2011).…”

A Self-Ensemble Approach for Partial Multi-Label Learning

“…Although be used to some extent, the label correlations cannot be confined to pairwise labels under realworld scenarios. The high-order method consider the correlations among label subsets or the whole class labels [18]- [20]. High-order methods could address more realistic label correlations, while it may suffer from high model complexities.…”

A novel multi-label classification algorithm based on K-nearest neighbor and random walk

“…Since SLC is merely a special case, MLC deals with a more difficult and general problem in the data mining domain, focusing on the following two challenges. (1) The number of label sets may be very large for test instances (e.g. being exponentially proportional to the total number of labels).…”

“…However, little progress has been made with these methods and algorithms [15][16][17][18][19][20][21][22][23][24][25][26][27][28][29] especially in the following aspects. (1) Complexity: in order to construct a graph model, a graph-based MLC method must map the instances of an entire training set to graph vertices, with many instances irrelevant to the test instances causing very high requirements for time and space and causing high computational complexity. (2) Heterogeneity: for similarity measurement among instances, nearly none of these methods consider the difference between discrete features and continuous features.…”