Incorporating Privileged Information Through Metric Learning

Abstract

In some pattern analysis problems, there exists expert knowledge, in addition to the original data involved in the classification process. The vast majority of existing approaches simply ignore such auxiliary (privileged) knowledge. Recently a new paradigm-learning using privileged information-was introduced in the framework of SVM+. This approach is formulated for binary classification and, as typical for many kernel-based methods, can scale unfavorably with the number of training examples. While speeding up training methods and extensions of SVM+ to multiclass problems are possible, in this paper we present a more direct novel methodology for incorporating valuable privileged knowledge in the model construction phase, primarily formulated in the framework of generalized matrix learning vector quantization. This is done by changing the global metric in the input space, based on distance relations revealed by the privileged information. Hence, unlike in SVM+, any convenient classifier can be used after such metric modification, bringing more flexibility to the problem of incorporating privileged information during the training. Experiments demonstrate that the manipulation of an input space metric based on privileged data improves classification accuracy. Moreover, our methods can achieve competitive performance against the SVM+ formulations.

Fast Algorithms for Linear and Kernel SVM+

“…After that, many variants of SVM+ have been proposed for solving different tasks [24,12,34,29,33,23]. In [24], Liang and Cherkassky developed a multi-task learning approach based on SVM+.…”

“…In [17], a multi-task multi-class extension of SVM+ was proposed. Fouad et al [12] designed a two-step approach for metric learning, and Xu et al [34] formulated a convex formulation for metric learning using privileged information based on the information theory metric learning (ITML) method. Sharmanska et al [29] proposed the Rank Transfer method for utilizing privileged information, and demonstrated the effectiveness of privileged information in various computer vision tasks.…”

Classifying Cognitive Profiles Using Machine Learning with Privileged Information in Mild Cognitive Impairment

“…In the GMLVQ framework, the privileged information is incorporated by fusing the metric Λ * in the privileged space 𝒳 * with the metric Λ in the original space 𝒳 (for more details, see Fouad et al, 2013). …”

“…We expect that the power of learning with privileged information will boost the classification performance, so that the classifier trained with CD as inputs, but able to incorporate fMRI indirectly in the training process ( M + -CD-PD), will have classification performance between the two extremes M-PD and M-CD, even though in the test phase, both M-CD and M + -CD-PD classify solely based on CD. The methodology is formulated in the framework of prototype-based classification (GMLVQ) with metric learning (Schneider et al, 2009; Schneider, 2010; Fouad et al, 2013). In this experiment, the original and privileged features correspond to cognitive profiles and brain imaging data, respectively.…”

Learning from the Mistakes of Others: Matching Errors in Cross-Dataset Learning

“…LUPI has been shown useful in a variety of learning scenarios such as ranking [28], categorization [37], structured prediction [9], data clustering [8], metric learning [10], face/gesture recognition [38], glaucoma detection [7], and recently learning with annotation disagreements [27]. Most LUPI methods (e.g.…”