We present an efficient probabilistic method for identity recognition in personal photo albums. Personal photos are usually taken under uncontrolled conditions -the captured faces exhibit significant variations in pose, expression and illumination that limit the success of traditional face recognition algorithms. We show how to improve recognition rates by incorporating additional cues present in personal photo collections, such as clothing appearance and information about when the photo was taken. This is done by constructing a Markov Random Field (MRF) that effectively combines all available contextual cues in a principled recognition framework. Performing inference in the MRF produces markedly improved recognition results in a challenging dataset consisting of the personal photo collections of multiple people. At the same time, the computational cost of our approach remains comparable to that of standard face recognition approaches.

This paper presents an algorithm for measuring hair and face appearance in 2D images. Our approach starts by using learned mixture models of color and location information to suggest the hypotheses of the face, hair, and background regions. In turn, the image gradient information is used to generate the likely suggestions in the neighboring image regions. Either Graph-Cut or Loopy Belief Propagation algorithm is then applied to optimize the resulting Markov network in order to obtain the most likely hair and face segmentation from the background. We demonstrate that our algorithm can precisely identify the hair and face regions from a large dataset of face images automatically detected by the state-of-the-art face detector.