Learning hierarchical multi-category text classification models

Abstract

We present a kernel-based algorithm for hierarchical text classification where the documents are allowed to belong to more than one category at a time. The classification model is a variant of the Maximum Margin Markov Network framework, where the classification hierarchy is represented as a Markov tree equipped with an exponential family defined on the edges. We present an efficient optimization algorithm based on incremental conditional gradient ascent in single-example subspaces spanned by the marginal dual variables. Experiments show that the algorithm can feasibly optimize training sets of thousands of examples and classification hierarchies consisting of hundreds of nodes. The algorithm's predictive accuracy is competitive with other recently introduced hierarchical multi-category or multilabel classification learning algorithms.

A semi-dependent decomposition approach to learn hierarchical classifiers

“…Hierarchical classification can, however, be seen as a whole rather than a series of local learning tasks; the idea being to optimize the global performance all at once. This approach is adopted in [4][5][6][7][8].…”

“…In [6], Rousu et al presented a kernel-based method in which the classification model is a variant of the maximum margin Markov network framework. This algorithm relies on a decomposition of the problem into single-example subproblems and conditional gradient ascent for optimisation of these subproblems.…”

“…Some of the papers cited above, for instance [3,6], compare their methods with two baseline algorithms: a kind of "flat" one-vs-rest multiclass SVM and a hierarchical classifier based on SVM, usually called H-SVM . Both algorithms consist in learning a binary classifier for each node (class) of the hierarchy to predict whether an example belongs to the class at that node or not.…”

“…However, this procedure may lead to inconsistent predictions with respect to T . To avoid these, a top-down prediction procedure can be used, as in [3,6]. Thus, an entry can only be assigned to a class j if it was previously classified into its parent class, par(j); therefore, an entry not assigned to one class, will automatically not be assigned to any of its descendants.…”

“…For instance, a news website, or a news service in general, needs to classify the latest articles into sections and subsections of the site [1][2][3][4][5][6]. This learning task is usually called hierarchical classification.…”