Automatic tagging and geotagging in video collections and communities

Larson, Martha; Soleymani, Mohammad; Serdyukov, Pavel; Rudinac, Stevan; Wartena, Christian; Murdock, Vanessa; Friedland, Gerald; Ordelman, Roeland; Jones, Gareth

doi:10.1145/1991996.1992047

Cited by 83 publications

(69 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…To overcome this challenge in related domains, automatic tagging mechanisms have been proposed that extract keywords from textual meta data and content. In the case of shared multimedia content, however, this is often not feasible with satisfying precision, as meta data can be sparse or ambiguous and concept detection from audio-visual signals is still considered more difficult than text-based alternatives [14]. For example, many videos on YouTube feature only a title and a brief textual description.…”

Section: Introductionmentioning

confidence: 99%

Exploiting User Comments for Audio-Visual Content Indexing and Retrieval

Eickhoff

Vries

2013

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. State-of-the-art content sharing platforms often require users to assign tags to pieces of media in order to make them easily retrievable. Since this task is sometimes perceived as tedious or boring, annotations can be sparse. Commenting on the other hand is a frequently used means of expressing user opinion towards shared media items. This work makes use of time series analyses in order to infer potential tags and indexing terms for audio-visual content from user comments. In this way, we mitigate the vocabulary gap between queries and document descriptors. Additionally, we show how large-scale encyclopaedias such as Wikipedia can aid the task of tag prediction by serving as surrogates for high-coverage natural language vocabulary lists. Our evaluation is conducted on a corpus of several million real-world user comments from the popular video sharing platform YouTube, and demonstrates significant improvements in retrieval performance.

show abstract

Section: Introductionmentioning

confidence: 99%

Exploiting User Comments for Audio-Visual Content Indexing and Retrieval

Eickhoff

Vries

2013

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…of metadata that can be used to index the content to facilitate its finding and re-finding. Tags can come in different forms including semantic tags, affective tags and geotags [2]. In contrast to classic tagging schemes where users direct input is mandatory, human-centered implicit tagging was proposed [3] to gather tags and annotations without any effort from users.…”

Section: Introductionmentioning

confidence: 99%

Multimedia implicit tagging using EEG signals

Soleymani

Pantić

2013

2013 IEEE International Conference on Multimedia and Expo (ICME)

Self Cite

View full text Add to dashboard Cite

Electroencephalogram (EEG) signals reflect brain activities associated with emotional and cognitive processes. In this paper, we demonstrate how they can be used to find tags for multimedia content without users' direct input. Alternative methods for multimedia tagging is attracting increasing interest from multimedia community. The new portable EEG helmets are paving the way for employing brain waves in human computer interaction. In this paper, we demonstrate the performance of EEG for tagging purposes using two different scenarios on MAHNOB-HCI database. First, an emotional tagging and classification using a reduced set of electrodes is presented. The emotional responses of 24 participants to short video clips are classified into three classes on arousal and valence. We show how a reduced set of electrodes based on previous studies can preserve and even enhance the emotional classification rate. We then demonstrate the feasibility of using EEG signals for tag relevance tasks. A set of images were shown to participants first, without any tag and then with a relevant or irrelevant tag. The relevance of the tag was assessed based on the EEG responses of the participants in the first second after the tag was depicted. Finally, we demonstrate that by aggregating multiple participants' responses we can significantly improve the tagging accuracy.

show abstract

“…The experiments are performed on two datasets, that are reissues of the TRECVID 2007, 2008 and 2009 data made for the purposes of the "Tagging Task: Professional Version" offered for the MediaEval 2010 3 benchmark [13]. This benchmark also provided ground truth in the form of semantic theme labels assigned by professional archivists.…”

Section: Datasetsmentioning

confidence: 99%

Leveraging visual concepts and query performance prediction for semantic-theme-based video retrieval

Rudinac

Larson

Hanjalic

2012

Int J Multimed Info Retr

Self Cite

View full text Add to dashboard Cite

In this paper, we present a novel approach that utilizes noisy shot-level visual concept detection to improve text-based video retrieval. As opposed to most of the related work in the field, we consider entire videos as the retrieval units and focus on queries that address a general subject matter (semantic theme) of a video. Retrieval is performed using a coherence-based query performance prediction framework. In this framework, we make use of video representations derived from the visual concepts detected in videos to select the best possible search result given the query, video collection, available search mechanisms and the resources for query modification. In addition to investigating the potential of this approach to outperform typical text-based video retrieval baselines, we also explore the possibility to achieve further improvement in retrieval performance through combining our concept-based query performance indicators with the indicators utilizing the spoken content of the videos. The proposed retrieval approach is data driven, requires no prior training and relies exclusively on the analyses of the video collection and different results lists returned for the given query text. The experiments are performed on the Media Eval 2010 datasets and demonstrate the effectiveness of our approach.

show abstract

Automatic tagging and geotagging in video collections and communities

Cited by 83 publications

References 35 publications

Exploiting User Comments for Audio-Visual Content Indexing and Retrieval

Exploiting User Comments for Audio-Visual Content Indexing and Retrieval

Multimedia implicit tagging using EEG signals

Leveraging visual concepts and query performance prediction for semantic-theme-based video retrieval

Contact Info

Product

Resources

About