Automatic Face Annotation in News Images by Mining the Web

Abstract

We consider the automatic annotation of faces of people mentioned in news. News stories provide a constant flow of potentially useful image indexing information, due to their huge diffusion on the web and to the involvement of human operators in selecting relevant images for the stories. In this work we investigate the possibility of actually exploiting this wealth of information.\ud We propose and evaluate a system for automatic face annotation of image news that is fully unsupervised and does not require any prior knowledge about topic or people involved. Key feature of our proposal is that it attempts to identify the essential piece of information---how a person with a given name looks like---by querying popular image search engines. Mining the web allows overcoming intrinsic limitations of approaches built above a predefined collection of stories: our system can potentially annotate people never handled before since its knowledge base is constantly expanded, as long as search engines keep on indexing the web. On the other hand, leveraging on image search engines forces to cope with the substantial amount of noise in search engine results. Our contribution shows experimentally that automatic face annotation may indeed be achieved based entirely on knowledge that lives in the web

Image auto-annotation with automatic selection of the annotation length

“…The resulting annotations length is not fixed, but depends on the selected threshold value t and varies for each query image. In this our approach differs from known methods such as Makadia et al (2008); Carneiro et al (2007) or even Medvet et al (2011) who also employ transfer functions.…”

“…An annotation transfer step very similar to Makadia et al (2008) has been recently proposed by Medvet et al (2011) to assign text-mined names to faces. They extend the original approach by weighting the considered name based on the nearest images distance to the query image and then transfer the name only if the weight surpasses a threshold.…”

“…Contrary to them we also address the problems occurring from a fixed annotation length by using weighted annotation and selecting terms with weights values greater than a threshold for transfer. We select a more general and simple transfer function than Makadia et al (2008)'s transfer schema and such following Medvet et al (2011)'s proposition. For further simplification we use only a single distance measure and a single feature set.…”

“…For further simplification we use only a single distance measure and a single feature set. Our method is designed towards solving the problem of choosing the assigned annotation's appropriate length-it is our methods's main advantage and novelty, neither addressed in Makadia et al (2008) (who use fixed annotation length) nor in Medvet et al (2011) (who face only a binary decision problem). Additionally we propose a transfer parameter optimization method which automatically tunes the resulting word count associated with the image, eliminating the need of manually setting the parameters, and such rendering our method de-facto parameter free.…”

“…Carneiro et al 2007). Recently another approach has been suggested by Makadia et al (2008) and picked up by Medvet et al (2011), that we adopted and extended in this paper.…”