Domain-Specific Face Synthesis for Video Face Recognition From a Single Sample Per Person

Abstract

In video surveillance, face recognition (FR) systems are employed to detect individuals of interest appearing over a distributed network of cameras. The performance of still-tovideo FR systems can decline significantly because faces captured in unconstrained operational domain (OD) over multiple video cameras have a different underlying data distribution compared to faces captured under controlled conditions in the enrollment domain with a still camera. This is particularly true when individuals are enrolled to the system using a single reference still. To improve the robustness of these systems, it is possible to augment the reference set by generating synthetic faces based on the original still. However, without the knowledge of the OD, many synthetic images must be generated to account for all possible capture conditions. FR systems may, therefore, require complex implementations and yield lower accuracy when training on many less relevant images. This paper introduces an algorithm for domain-specific face synthesis (DSFS) that exploits the representative intra-class variation information available from the OD. Prior to operation (during camera calibration), a compact set of faces from unknown persons appearing in the OD is selected through affinity propagation clustering in the captured condition space (defined by pose and illumination estimation). The domainspecific variations of these face images are then projected onto the reference still of each individual by integrating an imagebased face relighting technique inside the 3-D reconstruction framework. A compact set of synthetic faces is generated that resemble individuals of interest under the capture conditions relevant to the OD. In a particular implementation based on sparse representation classification, the synthetic faces generated with the DSFS are employed to form a cross-domain dictionary that accounts for structured sparsity, where the dictionary blocks combine the original and synthetic faces of each individual. Experimental results obtained with videos from the Chokepoint and COX-S2V data sets reveal that augmenting the reference gallery set of still-to-video FR systems using the proposed DSFS approach can provide a significantly higher level of accuracy compared with the state-of-the-art approaches, with only a moderate increase in its computational complexity.Index Terms-Face recognition, single sample per person, face synthesis, 3D face reconstruction, illumination transferring, sparse representation-based classification, video surveillance.

A Novel Neural Network Method for Face Recognition With a Single Sample Per Person

“…In [17], Multiple Convolutional Neural Network process the facial picture to produce several poses different characteristics. Multiple face representation techniques can deal with for minor changes and are therefore not effective when dealing in realistic implementations which have a lot of variations (e.g., pose, blurriness and extreme illumination) [18].…”

“…The generic intra-class variety might not be related to the gallery images, therefore it may not be necessary to extract features from the generic set. Furthermore, A huge number of photos gathered from outside data that include redundant data that could lead to complex in implementing and reduce the ability to deal with intra-class variations [18].…”

“…A lot of synthetic faces should be generated for all possible capture circumstances in the operational domain. Without choosing representative photos from the external data and the generating reference, synthetic images could need complicated developments [18].…”

Redundancy Removing Aggregation Network With Distance Calibration for Video Face Recognition

“…In addition to aggregating a video into a single 2D image, some schemes also leverage 3D modeling using a large range of pose variations. Mokhayeri et al [26] proposed a domain-specific face synthesizing technique to reconstruct the gallery dataset and project it under different illumination. For the schemes mentioned above, the fundamental challenge is to generate discriminative images from video clips.…”

A paired sparse representation model for robust face recognition from a single sample

“…State-of-the-art approaches designed to address SSPP problems in SRC-based FR systems can be roughly divided into three categories: (1) image patching methods, where the images are partitioned into several patches [8,9], (2) face synthesis technique to expand the gallery dictionary [10,11], and (3) generic learning methods, where a genetic training set 1 is used to leverage variational information from an auxiliary generic set of images to represent the differences between probe and gallery images [12,13]. Indeed, similar intra-class variations may be shared by different individuals in the generic set and reference regions of interest (ROIs) in the gallery.…”