Rethinking Summarization and Storytelling for Modern Social Multimedia

Rudinac, Stevan; Chua, Tat-Seng; Ferreyra, Nicolás E. Díaz; Friedland, Gerald; Gornostaja, Tatjana; Huet, Benoît; Kaptein, Rianne; Lindén, Krister; Moens, Marie Francine; Peltonen, Jaakko; Реди, Мириам; Schedl, Markus; Shamma, David A.; Smeaton, Alan F.; Xie, Lexing

doi:10.1007/978-3-319-73603-7_51

Cited by 7 publications

(5 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While most approaches on event detection center on social media data or news articles, the main focus of this paper is detection of urban micro-events in a real world data set containing extremely heterogeneous multimedia data. Our objective is therefore aligned with the recent efforts of multimedia community towards rethinking the very concept of event in the age of multimedia data that goes beyond simple text and visual modalities [31].…”

Section: Event Detectionmentioning

confidence: 99%

Multimodal Classification of Urban Micro-Events

Sukel

Rudinac

Worring

2019

Proceedings of the 27th ACM International Conference on Multimedia

Self Cite

View full text Add to dashboard Cite

In this paper we seek methods to effectively detect urban microevents. Urban micro-events are events which occur in cities, have limited geographical coverage and typically affect only a small group of citizens. Because of their scale these are difficult to identify in most data sources. However, by using citizen sensing to gather data, detecting them becomes feasible. The data gathered by citizen sensing is often multimodal and, as a consequence, the information required to detect urban micro-events is distributed over multiple modalities. This makes it essential to have a classifier capable of combining them. In this paper we explore several methods of creating such a classifier, including early, late, hybrid fusion and representation learning using multimodal graphs. We evaluate performance on a real world dataset obtained from a live citizen reporting system. We show that a multimodal approach yields higher performance than unimodal alternatives. Furthermore, we demonstrate that our hybrid combination of early and late fusion with multimodal embeddings performs best in classification of urban micro-events.

show abstract

Section: Event Detectionmentioning

confidence: 99%

Multimodal Classification of Urban Micro-Events

Sukel

Rudinac

Worring

2019

Proceedings of the 27th ACM International Conference on Multimedia

Self Cite

View full text Add to dashboard Cite

show abstract

“…One of the main challenges with analyzing social media data is the amount of content that needs to be processed. As recently noted by Rudinac et al [21], efficient summarization approaches are needed to facilitate exploration in such large and heterogeneous collections.…”

Section: Profilesmentioning

confidence: 99%

Interactive Search and Exploration in Discussion Forums Using Multimodal Embeddings

Gornishka

Rudinac

Worring

2019

MultiMedia Modeling

Self Cite

View full text Add to dashboard Cite

In this paper we present a novel interactive multimodal learning system, which facilitates search and exploration in large networks of social multimedia users. It allows the analyst to identify and select users of interest, and to find similar users in an interactive learning setting. Our approach is based on novel multimodal representations of users, words and concepts, which we simultaneously learn by deploying a general-purpose neural embedding model. We show these representations to be useful not only for categorizing users, but also for automatically generating user and community profiles. Inspired by traditional summarization approaches, we create the profiles by selecting diverse and representative content from all available modalities, i.e. the text, image and user modality. The usefulness of the approach is evaluated using artificial actors, which simulate user behavior in a relevance feedback scenario. Multiple experiments were conducted in order to evaluate the quality of our multimodal representations, to compare different embedding strategies, and to determine the importance of different modalities. We demonstrate the capabilities of the proposed approach on two different multimedia collections originating from the violent online extremism forum Stormfront and the microblogging platform Twitter, which are particularly interesting due to the high semantic level of the discussions they feature. CCS CONCEPTS• Information systems → Multimedia and multimodal retrieval. KEYWORDS multimedia analytics, search, exploration, interactive learning, multimodal embeddings, online discussion forums, social multimedia • First, compact but meaningful multimodal content representations are needed to ensure the interactivity of the system

show abstract

“…The automatic generation or automatic editing of media content is not a new concept. Examples vary from automatic generation of video summaries [8] to automatic "beautification of facial images [9].…”

Section: Gans To Generate Visual Mediamentioning

confidence: 99%

Challenges Associated with Generative Forms of Multimedia Content (Keynote Talk)

Smeaton

2019

2019 International Conference on Content-Based Multimedia Indexing (CBMI)

View full text Add to dashboard Cite

This short paper presents what is currently the main challenge associated with using Generative Adversarial Networks (GANs) to generate visual media. That challenge is around automatically determining the authenticity of an image/video, which is of increasing importance as fake videos and images start to proliferate on social media, especially when associated with political campaigning. The paper introduces GANs, outlines how they are used to generate visual media and summarises this major challenge.

show abstract

Rethinking Summarization and Storytelling for Modern Social Multimedia

Cited by 7 publications

References 46 publications

Multimodal Classification of Urban Micro-Events

Multimodal Classification of Urban Micro-Events

Interactive Search and Exploration in Discussion Forums Using Multimodal Embeddings

Challenges Associated with Generative Forms of Multimedia Content (Keynote Talk)

Contact Info

Product

Resources

About