Names and faces in the news

Abstract

We show quite good face clustering is possible for a dataset of inaccurately and ambiguously labelled face images. Our dataset is 44,773 face images, obtained by applying a face finder to approximately half a million captioned news images. This dataset is more realistic than usual face recognition datasets, because it contains faces captured "in the wild" in a variety of configurations with respect to the camera, taking a variety of expressions, and under illumination of widely varying color. Each face image is associated with a set of names, automatically extracted from the associated caption. Many, but not all such sets contain the correct name.We cluster face images in appropriate discriminant coordinates. We use a clustering procedure to break ambiguities in labelling and identify incorrectly labelled faces. A merging procedure then identifies variants of names that refer to the same individual. The resulting representation can be used to label faces in news images or to organize news pictures by individuals present.An alternative view of our procedure is as a process that cleans up noisy supervised data. We demonstrate how to use entropy measures to evaluate such procedures.

On Film Character Retrieval in Feature-Length Films

“…no text). The difference in the difficulty is apparent by comparing the examples in [6] with those used for evaluation in §3. For example, in [6] the face image size is restricted to be at least 86 × 86 pixels, whilst a significant number of faces we use are of lower resolution.…”

“…Fitzgibbon and Zisserman [14] investigated face clustering in feature films, though without explicitly using facial features for registration. Berg et al [6] consider the problem of clustering detected frontal faces extracted from web news pages. In a similar manner to us, affine registration with an underlying SVM-based facial feature detector is used for face rectification.…”

“…A related approach was described in [6]; alternative methods include pictorial structures [13], shape+appearance cascaded classifiers [15] and the method of Cristinacce et al [10]. We represent each facial feature, i.e.…”

Sentence Directed Video Object Codiscovery

“…Sentence semantics only provides ambiguous and implicit labels. This resembles another line of work that learns structured output from image captions (Berg et al 2004;Gupta and Davis 2008;Luo et al 2009;Jamieson et al 2010a, b;Plummer et al 2015;Mao et al 2016), treating the input as a parallel image-text dataset. However, all of these methods, except Gupta and Davis (2008) and Jamieson et al (2010a, b) use pretrained object models learned from other datasets.…”

Face Recognition and Retrieval in Video

“…Everingham et al [26,27] addressed the problem of automatically labeling faces of characters in TV or film materials with their names. Similar to the "Faces in the News" labeling in [16], where detected frontal faces in news images are tagged with names appearing in the news story text, they proposed to combine visual cues (face and cloth) and textual cues (subtitle and transcript) for assigning names. Regarding face processing [3], face detections in each frame are linked to derive face tracks, and each face is represented by local appearance descriptors computed around 13 facial features.…”