Visual interestingness in image sequences

Abstract

Interestingness is said to be the power of attracting or holding one's attention (because something is unusual or exciting, etc.). We, as humans, have the great capacity to direct our visual attention and judge the interestingness of a scene. Consider for example the image sequence in the figure on the right. The spider in front of the camera or the snow on the lens are examples of events that deviate from the context since they violate the expectations, and therefore are considered interesting. On the other hand, weather changes or a camera shift, do not raise human attention considerably, even though large regions of the image are influenced. In this work we firstly investigate what humans consider as "interesting" in image sequences. Secondly we propose a computer vision algorithm to automatically spot these interesting events. To this end, we integrate multiple cues inspired by cognitive concepts and discuss why and to what extent the automatic discovery of visual interestingness is possible. ABSTRACTInterestingness is said to be the power of attracting or holding one's attention (because something is unusual or exciting, etc.). 1 We, as humans, have the great capacity to direct our visual attention and judge the interestingness of a scene. Consider for example the image sequence in the figure on the right. The spider in front of the camera or the snow on the lens are examples of events that deviate from the context since they violate the expectations, and therefore are considered interesting. On the other hand, weather changes or a camera shift, do not raise human attention considerably, even though large regions of the image are influenced. In this work we firstly investigate what humans consider as "interesting" in image sequences. Secondly we propose a computer vision algorithm to automatically spot these interesting events. To this end, we integrate multiple cues inspired by cognitive concepts and discuss why and to what extent the automatic discovery of visual interestingness is possible. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. Categories and Subject Descriptors

Analyzing and Predicting GIF Interestingness

“…A content-based GIF interestingness method is developed and evaluated. We obtain a prediction performance that is comparable to the ones on still images [18,8,7]. The contributions of this work are as follows: (i) We analyze the effect of emotional attributes on GIF interestingness and provide an in-depth analysis into what makes a GIF interesting; (ii) We study the relationship between GIF interestingness to GIF popularity as well as image interestingness.…”

“…Gygli et al and Grabner et al [8,7] showed that visual content features related to unusualness, aesthetics and general preference are effective in predicting visual interestingness. More recently, Soleymani [18] built a model for personalized interest prediction for images.…”

Event photo mining from Twitter using keyword bursts and image clustering

“…The objective is to detect abnormal events such as invasions and accidents from fixed camera video streams. Recently, as its variant, detecting interesting events has been proposed [6]. They proposed a computational model which integrates multiple cues to evaluate visual interestingness of image sequences.…”

Wisdom of the Crowd in Egocentric Video Curation

“…those containing the user looking at his watch or mobile phone). Although much work has been done on determining "interestingness" ( [7]), it is far from solved as it requires good high-level understanding of the scene.…”