Jointly exploiting visual and non-visual information for event-related social media retrieval

Dao, Minh-Son; Boato, Giulia; Natale, Francesco G. B. De; Nguyen, Truc-Vien T.

doi:10.1145/2461466.2461494

Cited by 9 publications

(5 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such applications relate to methods that can detect event-related media and group them by the events they illustrate or refer to. From the end user's perspective, finding digital content related to social events is challenging, requiring to search large volumes of data, possibly at different sources and sites [151,161,174,45]. To this end, Iliakopoulou et al [86] proposed a multi-step multimedia retrieval framework that collects relevant and diverse multimedia content from multiple social media sources given an input news story or event of interest.…”

Section: Event Applicationsmentioning

confidence: 99%

Event-based media processing and analysis: A survey of the literature

Tzelepis

Mezaris

et al. 2016

Image and Vision Computing

Self Cite

View full text Add to dashboard Cite

Research on event-based processing and analysis of media is receiving an increasing attention from the scientific community due to its relevance for an abundance of applications, from consumer video management and video surveillance to lifelogging and social media. Events have the ability to semantically encode relationships of different informational modalities, such as visual-audio-text, time, involved agents and objects, with the spatio-temporal component of events being a key feature for contextual analysis. This unveils an enormous potential for exploiting new information sources and opening new research directions. In this paper, we survey the existing literature in this field. We extensively review the employed conceptualization of the notion of event in multimedia, the techniques for event representation and modeling, the feature representation and event inference approaches for the problems of event detection in audio, visual, and textual content. Furthermore, we review some key event-based multimedia applications, and various benchmarking activities that provide solid frameworks for measuring the performance of different event processing and analysis systems. We provide an in-depth discussion of the insights obtained from reviewing the literature and identify future directions and challenges.Keywords: Event-based media processing and analysis, event conceptualization, event representation and modeling, multimedia event detection, event-based applications and benchmarking, survey of the literature

show abstract

Section: Event Applicationsmentioning

confidence: 99%

Event-based media processing and analysis: A survey of the literature

Tzelepis

Mezaris

et al. 2016

Image and Vision Computing

Self Cite

View full text Add to dashboard Cite

show abstract

“…In [7] a multimodal clustering approach is proposed, which predicts the same cluster relationship by exploiting pairwise similarities for all different modalities and achieving supervised fusion of the heterogeneous features. The social event detection is transformed into a watershed-based image segmentation in [28] and [29], where visual and non-visual information are jointly exploited. A fully automated system for event recognition from an image gallery has been recently proposed in [30], by exploiting metadata information.…”

Section: A Event-based Media Analysismentioning

confidence: 99%

EventMask: A Game-Based Framework for Event-Saliency Identification in Images

Rosani

Boato

Natale

2015

IEEE Trans. Multimedia

Self Cite

View full text Add to dashboard Cite

The concept of "event" emerged in the last years as a key feature to efficiently index and retrieve media. Several approaches have been proposed to analyze the relationship between events and related media, enable event discovery, perform event-based media tagging, indexing, and retrieval. Despite the outstanding work done by several researchers in this area, a major problem that still remains open is how to infer the inherent link between visual concepts and events. In particular, the possibility of understanding which perceptual elements allow a human recognizing the event depicted by an image would for sure open new directions in event media discovery. In this paper we introduce the concept of Event Saliency to define the above event-revealing perceptual elements, and we propose an original method to detect it by exploiting crowd knowledge through gamification. We propose an adversarial game with a hidden purpose, where users are engaged in two competitive roles: masking photos to prevent competitors recognizing the related event, and discovering events in photos masked by other players. Rules and incentives are defined to minimize cheating and force players to focus on details that really matter. Finally, a suitable algorithm composes the masks created by different players on the same media, thus producing a saliency map that, differently from the traditional concept of saliency, does not focus on perceptual prominence but rather on event-related semantics of media. A thorough validation on two public dataset is presented, and initial experiments to apply event saliency in detection tasks are proposed. Furthermore, an event-saliency dataset is disclosed to allow further research in this domain.

show abstract

“…Early approaches for social event retrieval were highly tailored to the characteristics of a specific query and, thus, not applicable in a general context. For example, in [6] queries were manually extended by synonyms. Additionally, the authors employed external services that were manually selected depending on the requested event type in the user-provided query.…”

Section: Related Workmentioning

confidence: 99%

Social Event Mining in Large Photo Collections

Zaharieva

Zeppelzauer

Fabro

et al. 2015

Proceedings of the 5th ACM on International Conference on Multimedia Retrieval

View full text Add to dashboard Cite

A significant part of publicly available photos on the Internet depicts a variety of different social events. In order to organize this steadily growing media content and to make it easily accessible, novel indexing methods are required. Essential research questions in this context concern the efficient detection (clustering), classification, and retrieval of social events in large media collections. In this paper we explore two aspects of social events mining. First, the initial clustering of a given photo collection into single events and, second, the retrieval of relevant social events based on user queries. For both aspects we employ commonly available metadata information, such as user, time, GPS data, and user-generated textual descriptions. Performed evaluations in the context of social event detection demonstrate the strong generalization ability of our approach and the potential of contextual data such as time, user, and location. Experiments with social event retrieval clearly indicate the open challenge of mapping between previously detected event clusters and heterogeneous user queries.

show abstract

Jointly exploiting visual and non-visual information for event-related social media retrieval

Cited by 9 publications

References 19 publications

Event-based media processing and analysis: A survey of the literature

Event-based media processing and analysis: A survey of the literature

EventMask: A Game-Based Framework for Event-Saliency Identification in Images

Social Event Mining in Large Photo Collections

Contact Info

Product

Resources

About