Face Recognition in Videos by Label Propagation

Abstract

Abstract-We consider the problem of automatic identification of faces in videos such as movies, given a dictionary of known faces from a public or an alternate database. This has applications in video indexing, content based search, surveillance, and real time recognition on wearable computers. We propose a two stage approach for this problem. First, we recognize the faces in a video using a sparse representation framework using l1-minimization and select a few key-frames based on a robust confidence measure. We then use transductive learning to propagate the labels from the key-frames to the remaining frames by incorporating constraints simultaneously in temporal and feature spaces. This is in contrast to some of the previous approaches where every test frame/track is identified independently, ignoring the correlation between the faces in video tracks. Having a few key frames belonging to few subjects for label propagation rather than a large dictionary of actors reduces the amount of confusion. We evaluate the performance of our algorithm on Movie Trailer face dataset and five movie clips, and achieve a significant improvement in labeling accuracy compared to previous approaches.

Who’s that Actor? Automatic Labelling of Actors in TV Series Starting from IMDB Images

“…Pham et al [16] use label propagation to name people appearing in TV news broadcasts starting from transcripts for weak supervision. Kumar et al [17] also use label propagation to propagate actor labels from a set of key frames that are matched to a manually-curated selection of template images in a movie. However, in their case, and in semi-supervised learning in general, it is supposed that the source and target data are from the same distribution.…”

A Survey of the Trends in Facial and Expression Recognition Databases and Methods

“…al. [76] (2014): Sparse framework with l 1 -minimization is used for facial recognition in the IMFD [28] database for robustness to age, illumination, pose, expression, lighting and storage limitations in images extracted from videos. Two kinds of features are extracted: Scale invariant feature transform (SIFT) and local binary patterns.…”

Face Recognition Using a Hybrid SVM–LBP Approach and the Indian Movie Face Database

“…Therefore, since the system originally had 400 LBP vectors for the training database (2 per person), PCA reduces almost 95% the dimensions of the training database with a good recognition rate of 90.5%. Moreover, it has been shown that the RBF kernel (Figure 3 b) has the best recognition rates compared with other kernels and that a neighbourhood of (8,4) to compute the LBPs achieves prominent results (Figure 3 c).…”

“… The experiments are performed using the recent and challenging Indian Movie Face Database (IMFDB) 2 . There are few studies that deal with this database, due to its complexity and novelty 3,4 , and no previous reference has been found using the IMFDB with the LBP algorithm. Moreover, a comparison with related methods is included, as well.…”

“…The parameters to be chosen were the number of principal LBPs r, the neighbourhood of LBP (P, R) and the kernel function for SVM. When calculating the recognition rate for one parameter, the others were fixed, so that depending on the test the number of eigenvectors was 20, a radial basis function (RBF) kernel was selected and a neighbourhood (8,4) was used for the LBPs. Figure 3 graphically shows the mean recognition rates obtained from the experiments.…”