Supervised Mixed Norm Autoencoder for Kinship Verification in Unconstrained Videos

Abstract

Identifying kinship relations has garnered interest due to several applications such as organizing and tagging the enormous amount of videos being uploaded on the Internet. Existing research in kinship verification primarily focuses on kinship prediction with image pairs. In this research, we propose a new deep learning framework for kinship verification in unconstrained videos using a novel Supervised Mixed Norm regularization Autoencoder (SMNAE). This new autoencoder formulation introduces class-specific sparsity in the weight matrix. The proposed three-stage SMNAE based kinship verification framework utilizes the learned spatio-temporal representation in the video frames for verifying kinship in a pair of videos. A new kinship video (KIVI) database of more than 500 individuals with variations due to illumination, pose, occlusion, ethnicity, and expression is collected for this research. It comprises a total of 355 true kin video pairs with over 250,000 still frames. The effectiveness of the proposed framework is demonstrated on the KIVI database and six existing kinship databases. On the KIVI database, SMNAE yields video-based kinship verification accuracy of 83.18% which is at least 3.2% better than existing algorithms. The algorithm is also evaluated on six publicly available kinship databases and compared with best reported results. It is observed that the proposed SMNAE consistently yields best results on all the databases.

Reasoning Graph Networks for Kinship Verification: from Star-shaped to Hierarchical

“…In the past few years, many methods [10], [23]- [25] have been proposed for kinship verification and most of them pay attention to extracting discriminative features for each facial image. We can divide them into three categories: handcrafted approaches, distance metric-based approaches, and deep learning-based approaches.…”

A Unified Approach to Kinship Verification

“…An ensemble of four CNNs was applied to improve the accuracy. Kohl et al [28] applied autoencoders to detect kin similarity in unconstrained videos, using a supervised mixed-norm autoencoder to compute the sparse embeddings of the kin pairs. Deep autoencoders were also studied by Deadman et al [29] to detect informative facial features for kin verification, where a Sparse Discriminative Metric Loss (SDM-Loss) was derived to utilize the positive and negative training pairs.…”

Facial Kinship Verification: A Comprehensive Review and Outlook

“…In the anthropology and genetics domain, FKV can help to study the hereditary characteristics of close relatives in social relationships (M’charek, 2020 ). In the field of public social security, it can be applied to finding missing children, border control and customs, and criminal investigations (Kohli et al, 2019b ; Lu et al, 2014c ). In the social media domain, the FKV can be used for family photo album organization, improving the performance of face recognition systems and social media analysis (Lu et al, 2014a ).…”