Multi-label Classification with Output Kernels

Guo, Yuhong; Schuurmans, Dale

doi:10.1007/978-3-642-40991-2_27

Cited by 10 publications

(13 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In contrast, in joint SVM, the dependency between different tags are encoded in output kernels. In this sense, our work is also similar to LM-K [18] and M3L [4]. Interestingly, when the output kernel is linear, it is equivalent to the explicit relationship learning in MLRL.…”

Section: Related Workmentioning

confidence: 66%

“…To transform the pairwise and triplet-wise dependencies between tags into the inner product of two outputs containing those tags, 2-degree and 3-degree polynomial kernels are tried in [18] and it was reported that 2-degree is better than 3-degree. In [4,16,20], linear feature maps were exploited also for pairwise dependencies.…”

Section: Implicit and Explicit Linear Output Kernels On Tag-setsmentioning

confidence: 99%

“…Furthermore, another practical obstacle of most generative methods is the intractability of inference in tag prediction phase, therefore, usually some approximation techniques are applied. On the other hand, discriminative methods directly model the tag-predicting function, out of which TagProp [10], JEC [3] are metric-learning based approaches, rank-SVM [17], LM-K [18] are rank-learning based approaches, M3L [4] and Multi-Label Relationship Learning (MLRL) [16] are maximum-margin based approaches. One notable issue, and also difficulty, in discriminative methods is the dependencies between output tags, of which many state-of-the-art studies [4,11,18] have being aware.…”

Section: Related Workmentioning

confidence: 99%

“…On the other hand, discriminative methods directly model the tag-predicting function, out of which TagProp [10], JEC [3] are metric-learning based approaches, rank-SVM [17], LM-K [18] are rank-learning based approaches, M3L [4] and Multi-Label Relationship Learning (MLRL) [16] are maximum-margin based approaches. One notable issue, and also difficulty, in discriminative methods is the dependencies between output tags, of which many state-of-the-art studies [4,11,18] have being aware. In several recent studies [10,3,11], discriminative methods were reported to displayed empirically superior performance than generative ones on image annotation task.…”

Section: Related Workmentioning

confidence: 99%

See 3 more Smart Citations

Scalable, accurate image annotation with joint SVMs and output kernels

2015

View full text Add to dashboard Cite

a b s t r a c tThis paper studies how joint training of multiple support vector machines (SVMs) can improve the effectiveness and efficiency of automatic image annotation. We cast image annotation as an outputrelated multi-task learning framework, with the prediction of each tag's presence as one individual task. Evidently, these tasks are related via dependencies between tags. The proposed joint learning framework, which we call joint SVM, is superior to other related models in its impressive and flexible mechanisms in exploiting the dependencies between tags: first, a linear output kernel can be implicitly learned when we train a joint SVM; or, a pre-designed kernel can be explicitly applied by users when prior knowledge is available. Also, a practical merit of joint SVM is that it shares the same computational complexity as one single conventional SVM, although multiple tasks are solved simultaneously. Although derived from the perspective of multi-task learning, the proposed joint SVM is highly related to structured-output learning techniques, e.g. max-margin regression (Szedmak and Shawe-taylor [1]), structural SVM (Tsochantaridis [2]). According to our empirical results on several image-annotation benchmark databases, our joint training strategy of SVMs can yield substantial improvements, in terms of both accuracy and efficiency, over training them independently. In particular, it compares favorably with many other state-of-the-art algorithms. We also develop a "perceptron-like" online learning scheme for joint SVM to enable it to scale up better to huge data in real-world practice.

show abstract

Section: Related Workmentioning

confidence: 66%

Section: Implicit and Explicit Linear Output Kernels On Tag-setsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Scalable, accurate image annotation with joint SVMs and output kernels

2015

View full text Add to dashboard Cite

show abstract

“…Given the complexity of patterns captured by biological images, these shallow models of feature extraction may not be sufficient. Therefore, it is desirable to develop a multilayer feature extractor [7], [32], [33], alleviating the tedious process of manual feature engineering and enhancing the representation power.…”

Section: Introductionmentioning

confidence: 99%

Deep Model Based Transfer and Multi-Task Learning for Biological Image Analysis

Zhang

Zeng

et al. 2020

IEEE Trans. Big Data

134

View full text Add to dashboard Cite

A central theme in learning from image data is to develop appropriate representations for the specific task at hand. Thus, a practical challenge is to determine what features are appropriate for specific tasks. For example, in the study of gene expression patterns in Drosophila, texture features were particularly effective for determining the developmental stages from in situ hybridization images. Such image representation is however not suitable for controlled vocabulary term annotation. Here, we developed feature extraction methods to generate hierarchical representations for ISH images. Our approach is based on the deep convolutional neural networks that can act on image pixels directly. To make the extracted features generic, the models were trained using a natural image set with millions of labeled examples. These models were transferred to the ISH image domain. To account for the differences between the source and target domains, we proposed a partial transfer learning scheme in which only part of the source model is transferred. We employed multi-task learning method to fine-tune the pre-trained models with labeled ISH images. Results showed that feature representations computed by deep models based on transfer and multi-task learning significantly outperformed other methods for annotating gene expression patterns at different stage ranges.

show abstract

Bi-directional Representation Learning for Multi-label Classification

Guo

2014

Machine Learning and Knowledge Discovery in Databases

Self Cite

View full text Add to dashboard Cite

Multi-label classification is a central problem in many application domains. In this paper, we present a novel supervised bi-directional model that learns a low-dimensional mid-level representation for multilabel classification. Unlike traditional multi-label learning methods which identify intermediate representations from either the input space or the output space but not both, the mid-level representation in our model has two complementary parts that capture intrinsic information of the input data and the output labels respectively under the autoencoder principle while augmenting each other for the target output label prediction. The resulting optimization problem can be solved efficiently using an iterative procedure with alternating steps, while closed-form solutions exist for one major step. Our experiments conducted on a variety of multilabel data sets demonstrate the efficacy of the proposed bi-directional representation learning model for multi-label classification.

show abstract

Multi-label Classification with Output Kernels

Cited by 10 publications

References 28 publications

Scalable, accurate image annotation with joint SVMs and output kernels

Scalable, accurate image annotation with joint SVMs and output kernels

Deep Model Based Transfer and Multi-Task Learning for Biological Image Analysis

Bi-directional Representation Learning for Multi-label Classification

Contact Info

Product

Resources

About