Face Association across Unconstrained Video Frames Using Conditional Random Fields

Abstract

Automatic face association across unconstrained video frames has many practical applications. Recent advances in the area of object detection have made it possible to replace the traditional trackingbased association approaches with the more robust detection-based ones. However, it is still a very challenging task for real-world unconstrained videos, especially if the subjects are in a moving platform and at distances exceeding several tens of meters. In this paper, we present a novel solution based on a Conditional Random Field (CRF) framework. The CRF approach not only gives a probabilistic and systematic treatment of the problem, but also elegantly combines global and local features. When ambiguities in labels cannot be solved by using the face appearance alone, our method relies on multiple contextual features to provide further evidence for association. Our algorithm works in an on-line mode and is able to reliably handle real-world videos. Results of experiments using challenging video data and comparisons with other methods are provided to demonstrate the effectiveness of our method.

Uncertainty Modeling of Contextual-Connections Between Tracklets for Unconstrained Video-Based Face Recognition

“…Zheng et al [32] further introduced an end-to-end framework of a deep network with a CRF module for semantic segmentation. Du et al [7] used a CRF to solve the face association problem in unconstrained videos.…”

Unconstrained Still/Video-Based Face Verification with Deep Convolutional Neural Networks

“…In unconstrained scenarios, the camera can undergo abrupt movements, which makes persistent tracking a challenging task. Du et al proposed a conditional random field (CRF) framework for face association in two consecutive frames by utilizing the affinity of facial features, location, motion, and clothing appearance [32]. Our face association method utilizes the KLT tracker to track a face initiated from the face detection.…”

An Online Algorithm for Constrained Face Clustering in Videos

“…The dominant approach to face clustering (both in images and videos) is completely unsupervised, where the primary objective is to learn a suitable distance measure between the data samples [4,5,6,7]. Several methods [8,9] have proposed to use partial supervision to improve clustering performance. In the context of video-based clustering, significant improvement can be achieved by exploiting the temporal information about the occurrence of the faces [2].…”