Mixed bi-subject kinship verification via multi-view multi-task learning

Abstract

Bi-subject kinship verification addresses the problem of verifying whether there exists some kind of kin relationship (i.e., father-son, father-daughter, motherson and mother-daughter) between a pair of parent-child subjects based purely on their visual appearance. The task is challenging due to the involvement of two different subjects possibly with different genders and ages. In addition, collecting sufficient training samples for each type of kinship is difficult. In this work, we present a novel method to address these issues by considering each type of kin relation verification as one task and learning them at one time in the framework of multi-task learning, by sharing feature sets and useful structures among the tasks.Particularly our contributions are three folds: first, we introduce a new type of learning problem, called mixed bi-subject kinship verification, to the topic of bi-subject kinship verification: instead of simply verifying whether some fixed kinship relationship (e.g., mother-son) can be established for a given pair of parent-child images, we try to figure out whether any type of the four kinship relations can be established according to the visual features of the image pair, with no need to know the genders of the subjects to be verified beforehand. Second, we propose a novel multi-task learning method to address this problem with two transformation matrices -one is shared amongst all the tasks and the other is unique to each task. Both matrices are simultaneously learned in a joint framework, which enables our algorithm to utilize the common knowledge of the four tasks. Third, we propose a multi-view multi-task learning(MMTL) method to perform multiple feature fusion to improve the mixed bi-subject kinship verification performance. Extensive experiments on the large scale KinFaceW kinship database demonstrate the feasibility and effectiveness of the proposed algorithm.

Facial Kinship Verification: A Comprehensive Review and Outlook

“…It maps the distance metric space into a new metric space (Kulis, 2012 ). The commonly used basic distance metrics in kinship verification are Euclidean distance (Yan et al, 2014b ), Mahalanobis distance (Hu et al, 2014 , 2017 ; Kou et al, 2015 ; Lu et al, 2014c ; Wei et al, 2019 ; Yan et al, 2014a ; Zhang et al, 2015 ), bilinear similarity (Fang et al, 2016 ; Qin et al, 2016 ; Xu and Shang, 2016a , b ; Zhou et al, 2016a , b ), graph learning (Guo et al, 2014 ; Liang et al, 2018 ), cosine similarity (Yan et al, 2015 ; Yan, 2017 ), CCA (Lei et al, 2017 ) and other metric patterns (Liu et al, 2017 ; Liu and Zhu, 2017 ; Wu et al, 2018 ; Zhang et al, 2016 ; Zhao et al, 2018 ). Selective methods are illustrated in Fig.…”

“…Then the inputs of ESL are quadratic, which satisfies the inter- and intra- constraints on the similarity pattern for image pairs. Qin et al ( 2016 ) proposed a multitask-based bilinear similarity learning method. They combined the four kinship verification tasks to transfer the knowledge from one task to other tasks.…”

Co-evolutionary multi-task learning for dynamic time series prediction

“…Multitask learning has recently contributed to a number of successful real-world applications that gained better performance by exploiting shared knowledge for multi-task formulation. Some of these applications include 1) multi-task approach for " retweet" prediction behaviour of individual users [45], 2) recognition of facial action units [22], 3) automated Human Epithelial Type 2 (HEp-2) cell classification [46], 4) kin-relationship verification using visual features [47] and 5) object tracking [48].…”

Tensor cross-view quadratic discriminant analysis for kinship verification in the wild

“…Many researchers [1,2,3,4,5,6,7,8,9,10] used metric learning methods and have achieved reasonably good performance in kinship verification, but none of these methods tackle the kinship verification as a cross-view matching problem.…”