Face tracking and recognition with visual constraints in real-world videos

Abstract

We address the problem of tracking and recognizing faces in real-world, noisy videos. We track faces using a tracker that adaptively builds a target model reflecting changes in appearance, typical of a video setting. However, adaptive appearance trackers often suffer from drift, a gradual adaptation of the tracker to non-targets. To alleviate this problem, our tracker introduces visual constraints using a combination of generative and discriminative models in a particle filtering framework. The generative term conforms the particles to the space of generic face poses while the discriminative one ensures rejection of poorly aligned targets. This leads to a tracker that significantly improves robustness against abrupt appearance changes and occlusions, critical for the subsequent recognition phase. Identity of the tracked subject is established by fusing pose-discriminant and person-discriminant features over the duration of a video sequence. This leads to a robust video-based face recognizer with state-of-the-art recognition performance. We test the quality of tracking and face recognition on realworld noisy videos from YouTube as well as the standard Honda/UCSD database. Our approach produces successful face tracking results on over 80% of all videos without video or person-specific parameter tuning. The good tracking performance induces similarly high recognition rates: 100% on Honda/UCSD and over 70% on the YouTube set containing 35 celebrities in 1500 sequences.

Spatial-Temporal Knowledge Integration: Robust Self-Supervised Facial Landmark Tracking

“…YouTube Celebrities [20]: This dataset dataset constains 47 celebrities under varying poses, illumination and occlusion. Following the same setting as in [29], we evaluate STKI on six video clips annotated by [29].…”

Color Based New Algorithm for Detection and Single/Multiple Person Face Tracking in Different Background Video Sequence

“…Tabulated results indicate the robustness of integrated algorithm compared to CAMSHIFT alone. Celebrities dataset [23] videos were used for testing the algorithms discussed above. The algorithms are proficient of tracing moving lone face caught using a static camera; but, unable to track moving/unmoving single/multiple face captured using a moving camera.…”

“…The drawbacks and limitations of the works presented here, have helped in proposing our current novel algorithm. Proposed algorithm uses the videos that are part of the effort in literature [23] for testing. At the end, standard metrics [30][31][32] are used to measure the attainment of the work proposed.…”

“…We have considered ten YouTube Celebrities dataset [23] videos for testing. The videos belong to the different background categories discussed in section 1.…”

A Comprehensive Survey of Algorithms for Face Tracking in different Background Video Sequence

“…The video sequences being considered for testing as shown in Figures 6, 7, 11 and 13 are taken from Celebrities dataset [31]. The video sequence that is selected for testing as shown in Figure 8 is from Choke Point dataset [32].…”