Discriminative Methods for Multi-labeled Classification

Abstract

Abstract. In this paper we present methods of enhancing existing discriminative classifiers for multi-labeled predictions. Discriminative methods like support vector machines perform very well for uni-labeled text classification tasks. Multi-labeled classification is a harder task subject to relatively less attention. In the multi-labeled setting, classes are often related to each other or part of a is-a hierarchy. We present a new technique for combining text features and features indicating relationships between classes, which can be used with any discriminative algorithm. We also present two enhancements to the margin of SVMs for building better models in the presence of overlapping classes. We present results of experiments on real world text benchmark datasets. Our new methods beat accuracy of existing methods with statistically significant improvements.

Classifier Chains for Multi-label Classification

“…A multi-label dataset D is therefore composed of n examples (x 1 , S 1 ), (x 2 , S 2 ), · · · , (x n , S n ). The multilabel problem is receiving increased attention and is relevant to many domains such as text classification [10,2], and genomics [19,16].…”

“…In this fashion, a single-label classifier can be employed to make single-label classifications, and these are then transformed back into multi-label representations. Prior problem transformation approaches have employed algorithms such as Support Vector Machines [2], Naive Bayes [5] and k Nearest Neighbor methods [19].…”

“…The most well known problem transformation method is the binary relevance method (BM) [13,2,19]. BM transforms any multi-label problem into one binary problem for each label.…”

“…The consensus view in the literature is that it is crucial to take into account label correlations during the classification process [3,2,15,8,11,18,4]. However as the size of multi-label datasets grows, most methods struggle with the exponential growth in the number of possible correlations.…”

Expressive Power of Binary Relevance and Chain Classifiers Based on Bayesian Networks for Multi-label Classification

“…Each classifier /$ is learned from predictor variables and C$ data, and the results are combined to form multi-label prediction. This method, called binary relevance [6], is easily implementable, has low computational complexity and is fully parallelizable. Hence it is scalable to a large number of classes.…”