An Automatic System for Unconstrained Video-Based Face Recognition

Abstract

Although deep learning approaches have achieved performance surpassing humans for still image-based face recognition, unconstrained video-based face recognition is still a challenging task due to large volume of data to be processed and intra/inter-video variations on pose, illumination, occlusion, scene, blur, video quality, etc. In this work, we consider challenging scenarios for unconstrained video-based face recognition from multiple-shot videos and surveillance videos with low-quality frames. To handle these problems, we propose a robust and efficient system for unconstrained video-based face recognition, which is composed of modules for face/fiducial detection, face association, and face recognition. First, we use multi-scale single-shot face detectors to efficiently localize faces in videos. The detected faces are then grouped respectively through carefully designed face association methods, especially for multi-shot videos. Finally, the faces are recognized by the proposed face matcher based on an unsupervised subspace learning approach and a subspace-to-subspace similarity metric. Extensive experiments on challenging video datasets, such as Multiple Biometric Grand Challenge (MBGC), Face and Ocular Challenge Series (FOCS), IARPA Janus Surveillance Video Benchmark (IJB-S) for low-quality surveillance videos and IARPA JANUS Benchmark B (IJB-B) for multiple-shot videos, demonstrate that the proposed system can accurately detect and associate faces from unconstrained videos and effectively learn robust and discriminative features for recognition.Index Terms-Unconstrained video-based face recognition, face tracking, face association. !

Uncertainty Modeling of Contextual-Connections Between Tracklets for Unconstrained Video-Based Face Recognition

“…max: similarity is computed by the maximum of all frame-wise cosine similarities between a gallery and a tracklet, or two tracklets. On IJB-S, we also implement the subspace-based similarity following [31], denoted as sub.…”

“…UGG-U(favg) directly uses the cosine similarities between averageflattened features. UGG-U(sub) uses the subspace-subspace similarity proposed in[31].…”

Accurate Person Identification Based on Combined Sit-to-Stand and Stand-to-Sit Movements Measured Using Doppler Radars

“…UTOMATIC person identification enables monitoring and authentication in security applications. Face recognition using cameras has been the most widely used method for identification [1], [2], and other devices such as smart speakers have become ubiquitous and can be used for identification [3]. However, these devices present privacy issues.…”

Video-Based Face Recognition

“…Face sets/sequences are matched under a certain similarity metric based on the face features in sets/sequences. Figure 2 illustrates an example of videobased face recognition system proposed in [2]. It exactly consists of four steps: (1) face detection from input videos and alignment by a face detector, (2) face feature extraction by a face classifier, (3) face association across video frames by single/multiple-shot association techniques, and (4) feature sets modeling and matching by subspace learning and subspace-based similarity metrics.…”