Synthetic face generation under various operational conditions in video surveillance

2015 IEEE Symposium Series on Computational Intelligence

Sabourin

et al. 2015

Systems for still-to-video face recognition (FR) are typically used to detect target individuals in watch-list screening applications. These surveillance applications are challenging because the appearance of faces changes according to capture conditions, and very few reference stills are available a priori for enrollment. To improve performance, an adaptive appearance model tracker (AAMT) is proposed for on-line learning of a track-face-model linked to each individual appearing in the scene. Meanwhile, these models are matched over successive frames against stored gallery-face-models, extracted from reference still images of each target individual (enrolled to the system) for robust spatiotemporal FR. In addition, compared to the gallery-face-models produced by selfupdating FR systems, the track-face-models (produced by the AAMT-FR system) are updated from facial captures that are more reliably selected, and can incorporate greater intra-class variations from the operational environment. Track-facemodels allow selecting facial captures for modeling more reliably than self-updating FR systems, and can incorporate a greater diversity of intra-class variation from the operational environment. Performance of the proposed approach is compared with several state-of-the-art FR systems on videos from the Chokepoint dataset when a single reference template per target individual is stored in the gallery. Experimental results show that the proposed system can achieve a significantly higher level of FR performance, especially when the diverse facial appearances captured through AAMT correspond to that of reference stills.

Section: Backgound -Still-to-video Frmentioning

confidence: 99%

Video Face Recognition From A Single Still Image Using an Adaptive Appearance Model Tracker

Dewan

2015 IEEE Symposium Series on Computational Intelligence

Sabourin

et al. 2015

“…In multiple face representations, different feature extraction techniques are employed to generate multiple discriminant and robust representations from a single reference still [6], where the key issue with this type of approaches is combining those representations appropriately. In synthetic face generation, several virtual face images are synthesized using 2D morphing or 3D reconstructions to enhance the number of target samples with different pose and viewpoints [27], [39]. The problem with these approaches is to exploit prior knowledge to locate the facial components reliably.…”

Section: Single Sample Per Person Solutionsmentioning

confidence: 99%

“…In pattern recognition literature [46], this challenging situation is referred to as a "single sample per person" (SSPP) problem, and the resulting lack of representativeness of facial models can yield poor FR performance. Techniques specialized for a SSPP problem in FR rely on multiple face representations (employing various face descriptors), synthetic generation (2D morphing and 3D reconstruction), and enlarging the training set using auxiliary data [28], [46], [39]. In this paper, the SSPP problem found in watch-list screening is addressed by exploiting multiple face representations, particularly through multiple patch configurations and multiple feature extraction techniques to design the individual-specific classification system.…”

Section: Introductionmentioning

confidence: 99%

Robust watch-list screening using dynamic ensembles of SVMs based on multiple face representations

Bashbaghi

Machine Vision and Applications

Sabourin

et al. 2017

Self Cite

Abstract. Still-to-video face recognition (FR) is an important function in video surveillance, where faces captured over a network of video cameras are matched against reference stills of target individuals. Screening faces against a watch-list is a challenging video surveillance application because the appearance of faces vary due to changing capture conditions and operational domains. The facial models used for matching may not be representative of faces captured with video cameras because they are typically designed a priori with only one reference still. In this paper, a multi-classifier framework is proposed for robust still-to-video FR based on multiple and diverse face representations of a single reference face still. During enrollment of a target individual, the single reference face still is modeled using an ensemble of SVM classifiers based on different patches and face descriptors. Multiple feature extraction techniques are applied to patches isolated in the reference still to generate a diverse SVM pool that provides robustness to common nuisance factors (e.g., variations in illumination and pose). The estimation of discriminant feature subsets, classifier parameters, decision thresholds, and ensemble fusion functions is achieved using the high-quality reference still and a large number of faces captured in lower quality video of non-target individuals in the scene. During operations, the most competent subset of SVMs are dynamically selected according to capture conditions. Finally, a head-face tracker gradually regroups faces captured from different people appearing in a scene, while each individual-specific ensemble performs face matching. The accumulation of matching scores per face track leads to a robust spatio-temporal FR when accumulated ensemble scores surpass a detection threshold. Experimental results obtained with the Chokepoint and COX-S2V datasets show a significant improvement in performance w.r.t. reference systems, especially when individual-specific ensembles (1) are designed using exemplar-SVMs rather than one-class SVMs, and (2) exploit score-level fusion of local SVMs (trained using features extracted from each patch), rather than using either decision-level or feature-level fusion with a global SVM (trained by concatenating features extracted from patches).

“…Multiple virtual views are synthesized by linear shape prediction [29], mesh warping [30], morphing [31], symmetry property [32], partitioning a face in several sub-images [33], affine transformation [34], noise perturbation [35], shifting [36], and active appearance model [37]. A recurring problem with the synthetic generation is that they need to locate facial components reliably to determine the pose angle for pose compensation.…”

Section: Sspp Techniques For Still-to-video Face Recognitionmentioning

confidence: 99%

Adaptive appearance model tracking for still-to-video face recognition

Dewan

Marcialis

et al. 2016

Pattern Recognition

Systems for still-to-video face recognition (FR) seek to detect the presence of target individuals based on reference facial still images or mug-shots. These systems encounter several challenges in video surveillance applications due to variations in capture conditions (e.g., pose, scale, illumination, blur and expression) and to camera inter-operability. Beyond these issues, few reference stills are available during enrollment to design representative facial models of target individuals. Systems for still-to-video FR must therefore rely on adaptation, multiple face representation, or synthetic generation of reference stills to enhance the intra-class variability of face models. Moreover, many FR systems only match high quality faces captured in video, which further reduces the probability of detecting target individuals. Instead of matching faces captured through segmentation to reference stills, this paper exploits Adaptive Appearance Model Tracking (AAMT) to gradually learn a track-face-model for each individual appearing in the scene. The Sequential Karhunen-Loeve technique is used for online learning of these track-face-models within a particle filter-based face tracker. Meanwhile, these models are matched over successive frames against the reference still images of each target individual enrolled to the system, and then matching scores are accumulated over several frames for robust spatiotemporal recognition. A target individual is recognized if scores accumulated for a track-face-model over a fixed time surpass some decision threshold. The main advantage of AAMT over traditional still-to-video FR systems is the greater diversity of facial representation that may be captured during operations, and this can lead to better discrimination for spatiotemporal recognition. Compared to state-of-the-art adaptive biometric systems, the proposed method selects facial captures to update an individual's face model more reliably because it relies on information from tracking. Simulation results obtained with the Chokepoint video dataset indicate that the proposed method provides a significantly higher level of performance compared state-of-the-art systems when a single reference still per individual is available for matching. This higher level of performance is achieved when the diverse facial appearances that are captured in video through AAMT correspond to that of reference stills.