SHIATSU: tagging and retrieving videos without worries

Abstract

The dramatic growth of video content over modern media channels (such as the Internet and mobile phone platforms) directs the interest of media broadcasters towards the topics of video retrieval and content browsing. Several video retrieval systems benefit from the use of semantic indexing based on content, since it allows an intuitive categorization of videos. However, indexing is usually performed through manual annotation, thus introducing potential problems such as ambiguity, lack of information, and non-relevance of index terms. In this paper, we present SHIATSU, a complete system for video retrieval which is based on the (semi-)automatic hierarchical semantic annotation of videos exploiting the analysis of visual content; videos can then be searched by means of attached tags and/or visual features. We experimentally evaluate the performance of SHIATSU on two different real video benchmarks, proving its accuracy and efficiency.

Semantics discovery in social tagging systems: A review

“…Mutual reinforcement principle has been proposed which states, "two tags are deemed similar if they have been associated to similar resources, and vice-versa that is, two resources are deemed similar if they have been labelled by similar tags", in order to compute tag and resource similarity in large-scale folksonomies. SHIATSU is a system developed by Bartolini et al [10] for automatic suggestion of user labels for videos at the shot level. SHIATSU is based on the opinion that the objects that share similar visual content also have the same semantic content.…”

“…The drawback identified by [10] regarding co-occurrence is that tag co-occurrence is not a solution of homonymy/polysemy problem when used alone.…”

A "smart" multimedia guide for indoor contextual navigation in cultural heritage applications

“…This final step amounts to solve an instance of the Maximum Weight Clique Problem on a small graph. Note that, while for objects of type image tags are directly associated to images, when annotating videos, we are able to predict tags not only for shots but even for videos, by opportunely propagate tags at the shot level to the video level [5]. Given a user-provided set of tags, as query semantic concepts, objects are selected by the query processor by applying a co-occurrence-based distance function d S on T .…”

Recommending Multimedia Objects in Cultural Heritage Applications

“…Then a "biased" average d F is used to aggregate distance values of matched elements. Videos are first segmented into shots [9]. Then, each shot o is represented by a single representative key frame (e.g., the first frame of the shot), so that shots can be compared by means of the above image similarity function d F .…”

“…This final step amounts to solve an instance of the Maximum Weight Clique Problem on a small graph [10]. Note that, while for objects of type image tags are directly associated to images, when annotating videos, we are able to predict tags not only for shots but even for videos, by opportunely propagate tags at the shot level to the video level [9].…”