Distance Metric Learning Using Privileged Information for Face Verification and Person Re-Identification

Abstract

In this paper, we propose a new approach to improve face verification and person re-identification in the RGB images by leveraging a set of RGB-D data, in which we have additional depth images in the training data captured using depth cameras such as Kinect. In particular, we extract visual features and depth features from the RGB images and depth images, respectively. As the depth features are available only in the training data, we treat the depth features as privileged information, and we formulate this task as a distance metric learning with privileged information problem. Unlike the traditional face verification and person re-identification tasks that only use visual features, we further employ the extra depth features in the training data to improve the learning of distance metric in the training process. Based on the information-theoretic metric learning (ITML) method, we propose a new formulation called ITML with privileged information (ITML+) for this task. We also present an efficient algorithm based on the cyclic projection method for solving the proposed ITML+ formulation. Extensive experiments on the challenging faces data sets EUROCOM and CurtinFaces for face verification as well as the BIWI RGBD-ID data set for person re-identification demonstrate the effectiveness of our proposed approach.

Fast Algorithms for Linear and Kernel SVM+

“…After that, many variants of SVM+ have been proposed for solving different tasks [24,12,34,29,33,23]. In [24], Liang and Cherkassky developed a multi-task learning approach based on SVM+.…”

“…In [17], a multi-task multi-class extension of SVM+ was proposed. Fouad et al [12] designed a two-step approach for metric learning, and Xu et al [34] formulated a convex formulation for metric learning using privileged information based on the information theory metric learning (ITML) method. Sharmanska et al [29] proposed the Rank Transfer method for utilizing privileged information, and demonstrated the effectiveness of privileged information in various computer vision tasks.…”

Person Reidentification Using Deep Convnets With Multitask Learning

“…In contrast to the above supervised methods, side information, which can be collected in an unsupervised manner and indicates that certain examples belong to the same class, can be used by the relevance component analysis (RCA) method to learn a Mahalanobis metric [5]. Additional information, such as depth information, can be used with a modified version of information theoretic metric learning [11] to improve re-identification performance [59]. The fact that people move through camera networks has been used to learn multiple related Mahalanobis distance metrics between camera pairs [42], however, this approach required knowledge of the camera network layout and different training for each camera pair.…”

Visual Recognition by Learning From Web Data via Weakly Supervised Domain Generalization

“…In the LUPI paradigm, the training samples are associated with additional features that are not available for the testing data, which are referred to as PI. In some recent works [9], [39]- [41], PI was exploited for different computer vision tasks. In [39], a rank SVM method was proposed to rank Web images based on PI.…”

“…In [39], a rank SVM method was proposed to rank Web images based on PI. In [40] and [41], PI was incorporated into distance metric learning. However, these works assume that the training data and the testing data are with the same data distribution, while this assumption does not hold in our setting.…”