Understanding and predicting importance in images

Abstract

What do people care about in an image? To drive computational visual recognition toward more human-centric outputs, we need a better understanding of how people perceive and judge the importance of content in images. In this paper, we explore how a number of factors relate to human perception of importance. Proposed factors fall into 3 broad types: 1) factors related to composition, e.g. size, location, 2) factors related to semantics, e.g. category of object or scene, and 3) contextual factors related to the likelihood of attribute-object, or object-scene pairs. We explore these factors using what people describe as a proxy for importance. Finally, we build models to predict what will be described about an image given either known image content, or image content estimated automatically by recognition systems.

What Makes a Photograph Memorable?

“…As opposed to other image properties, there are no previous studies that try to quantify individual, everyday photos in terms of how memorable they are, and there are no computer vision systems that try to predict image memorability. This is contrary to many other photographic properties that have been addressed in the literature such as photo quality [8], aesthetics [9], [10], interestingness [11], saliency [12], attractiveness [13], composition [14], [15], color harmony [16], and importance [17], [18]. Also, there are no databases of photographs calibrated in terms of the degree of memorability of each image.…”

“…This work is a first attempt to quantify this important property of individual images. Future work will investigate the relationship between image memorability and other measures such as object importance [17], [18], saliency [12], and photo quality [8].…”

“…as well as demographics and appearance of people (e.g., clothing, accessories, race, gender, etc.). Please see [18] for details.…”

A Max-Margin Riffled Independence Model for Image Tag Ranking

“…We consider a wider range of image tags including both objects and attributes. Berg et al [1] consider prediction of importance for objects, scenes and attributes, and study the influence of relations among objects in the same scene on people's perception of importance in images, which is similar to our goals. However, instead of modeling importance prediction as a binary decision problem, we predict image tags with multiple importance levels.…”

“…Predicting the importance of image content is a challenging problem in computer vision and has been addressed by relatively small amount of work [21,1]. Spain and Perona [21] take an object-centric stance, predicting the importance of objects in an image.…”