Video-Based Framework for Face Recognition in Video

Abstract

This paper presents a number of new views and techniques claimed to be very important for the problem of face recognition in video (FRiV). First, a clear differentiation is made between photographic facial data and video-acquired facial data as being two different modalities: one providing hard biometrics, the other providing softer biometrics. Second, faces which have the resolution of at least 12 pixels between the eyes are shown to be recognizable by computers just as they are by humans. As a way to deal with low resolution and quality of each individual video frame, the paper offers to use the neuro-associative principle employed by human brain, according to which both memorization and recognition of data are done based on a flow of frames rather than on one frame: synaptic plasticity provides a way to memorize from a sequence, while the collective decision making over time is very suitable for recognition of a sequence. As a benchmark for FRiV approaches, the paper introduces the IIT-NRC video-based database of faces which consists of pairs of low-resolution video clips of unconstrained facial motions. The recognition rate of over 95%, which we achieve on this database, as well as the results obtained on real-time annotation of people on TV allow us to believe that the proposed framework brings us closer to the ultimate benchmark for the FRiV approaches, which is "if you are able to recognize a person, so should the computer".

Super-resolution of faces using the epipolar constraint

“…The SR scheme has first been applied to a video sequence from the NRC-IIT Facial Video Database [8], which contains compressed video sequences of size 160x120 pixels at 20 fps recorded by a webcam. The point spread function g was chosen as a Gaussian with size 7x7 and standard deviation 1.5.…”

Face Recognition and Retrieval in Video

“…Their approach performs well on the data set of 16 subjects and the UCSD/Honda database [64]. Gorodnichy [43] proposed to use the neuro-associative principle for face recognition, according to which both memorization and recognition are done based on a flow of frames. The temporal dependence between consecutive images is considered by adding extra neurons.…”

“…Method Key-frame based Approaches [90], [40], [47], [114], [100], [17], [115], [31], [78], [85], [98], [101], [118] Temporal Model based Approaches [74], [73], [72], [75], [18], [24], [67], [69], [68], [122], [120], [123], [121], [64], [65], [66], [79], [55], [2], [43], [50], [49] Image-Set Matching based Approaches Statistical model-based [93], [4], [96], [7], [10], [6], [9] Mutual subspace-based [110], [90], [35], [82], [108], [56], [57], [5]<...>…”

Comparison of ARTMAP Neural Networks for Classification for Face Recognition from Video

“…The dataset used for computer simulations was collected by the National Research Council (NRC) [8]. It contains 22 Table 1.…”

“…The data set has been collected by the National Research Council of Canada [8], and corresponds to video sequences that display the face of a single person under different scenarios such as partial occlusion, pose, facial expression, motion, resolution and proximity. The average performance is assessed in terms of resources required during training and the generalization error during operation.…”