Generating “Visual Clouds” from Multiplex Networks for TV News Archive Query Visualization

Ren, Haolin; Renoust, Benjamin; Viaud, Marie-Luce; Mélançon, Guy; Satoh, Shin’ichi

doi:10.1109/cbmi.2018.8516482

“…Within digital humanities fields, such as digital cultural heritage, archaeology and data journalism, many multilayer approaches [vV17, MDG16, DHRL*12, SGo16, RRV*18, MBvL*17] can be found. Digital access to source texts and natural language processing techniques such as Named‐Entity Recognition and Topic Modelling allow for vast Digital Humanities data sets to be built [MDG16].…”

Section: Multilayer Network and Related Conceptsmentioning

confidence: 99%

The State of the Art in Multilayer Network Visualization

McGee

¹

,

Ghoniem

²

,

Mélançon

³

et al. 2019

Computer Graphics Forum

View full text Add to dashboard Cite

Modelling relationship between entities in real‐world systems with a simple graph is a standard approach. However, reality is better embraced as several interdependent subsystems (or layers). Recently, the concept of a multilayer network model has emerged from the field of complex systems. This model can be applied to a wide range of real‐world data sets. Examples of multilayer networks can be found in the domains of life sciences, sociology, digital humanities and more. Within the domain of graph visualization, there are many systems which visualize data sets having many characteristics of multilayer graphs. This report provides a state of the art and a structured analysis of contemporary multilayer network visualization, not only for researchers in visualization, but also for those who aim to visualize multilayer networks in the domain of complex systems, as well as those developing systems across application domains. We have explored the visualization literature to survey visualization techniques suitable for multilayer graph visualization, as well as tools, tasks and analytic techniques from within application domains. This report also identifies the outstanding challenges for multilayer graph visualization and suggests future research directions for addressing them.

show abstract

“…Character networks then became a natural focus for story analysis which from literature (Knuth 1993;Waumans et al 2015;Chen et al 2019) expanded to multimedia content (Weng et al 2009;Tan et al 2014;Tran and Jung 2015;Mish 2016;He et al 2018). Particular attention has been paid to dialogue structure (Park et al 2012;Gorinski and Lapata 2018), which leads to an extension of network modeling to multilayer models (Lv et al 2018;Ren et al 2018;Mourchid et al 2018).…”

Section: Related Workmentioning

confidence: 99%

“…It provides an automatic character interaction network construction and story segmentation by combining both visual and subtitle features. In the Visual Clouds (Ren et al 2018) networks extracted from TV news videos are used as a backbone support for interactive search refinement on heterogeneous data. However, layers cannot be investigated individually.…”

Section: Related Workmentioning

confidence: 99%

“…A movie script assembles all movie elements in a temporal fashion (scenes, dialogues) and highlights specific information such as characters and setting details, so that it supports automatic movie analysis (Jhala 2008;Mourchid et al 2018). In recent years, image analysis tools have tremendously enhanced our automatic understanding of image content (Guo et al 2016), and although tasks such as picture localization remain challenging (Demirkesen and Cherifi 2008;Pastrana-Vidal et al 2006), we may enrich textual approaches with face detection and recognition (Jiang and Learned-Miller 2017;Cao et al 2018) or with scene description (Johnson et al 2016;Yang et al 2017).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Movienet: a movie multilayer network model using visual and textual semantic cues

Mourchid

¹

,

Renoust

²

,

Roupin

³

et al. 2019

Self Cite

View full text Add to dashboard Cite

Discovering content and stories in movies is one of the most important concepts in multimedia content research studies. Network models have proven to be an efficient choice for this purpose. When an audience watches a movie, they usually compare the characters and the relationships between them. For this reason, most of the models developed so far are based on social networks analysis. They focus essentially on the characters at play. By analyzing characters interactions, we can obtain a broad picture of the narration's content. Other works have proposed to exploit semantic elements such as scenes, dialogues, etc.. However, they are always captured from a single facet. Motivated by these limitations, we introduce in this work a multilayer network model to capture the narration of a movie based on its script, its subtitles, and the movie content. After introducing the model and the extraction process from the raw data, we perform a comparative analysis of the whole 6-movie cycle of the Star Wars saga. Results demonstrate the effectiveness of the proposed framework for video content representation and analysis.

show abstract

“…Character networks then became a natural focus for story analysis which from literature [38,3,39] expanded to multimedia content [40,4,41,5,7,42]. Particular attention has been paid to dialogue structure [2,43], which leads to an extension of network modeling to multilayer models [44,45,8].…”

Section: Related Workmentioning

confidence: 99%

Movienet: A Movie Multilayer Network Model using Visual and Textual Semantic Cues

Mourchid¹,

Renoust²,

Roupin³

et al. 2019

Preprint

Self Cite

0

View full text Add to dashboard Cite

Discovering content and stories in movies is one of the most important concepts in multimedia content research studies. Network models have proven to be an efficient choice for this purpose. When an audience watches a movie, they usually compare the characters and the relationships between them. For this reason, most of the models developed so far are based on social networks analysis. They focus essentially on the characters at play. By analyzing characters interactions, we can obtain a broad picture of the narration's content. Other works have proposed to exploit semantic elements such as scenes, dialogues, etc.. However, they are always captured from a single facet. Motivated by these limitations, we introduce in this work a multilayer network model to capture the narration of a movie based on its script, its subtitles, and the movie content. After introducing the model and the extraction process from the raw data, we perform a comparative analysis of the whole 6-movie cycle of the Star Wars saga. Results demonstrate the effectiveness of the proposed framework for video content representation and analysis.

show abstract

Generating “Visual Clouds” from Multiplex Networks for TV News Archive Query Visualization

Cited by 8 publications

References 27 publications

The State of the Art in Multilayer Network Visualization

The State of the Art in Multilayer Network Visualization

Movienet: a movie multilayer network model using visual and textual semantic cues

Movienet: A Movie Multilayer Network Model using Visual and Textual Semantic Cues

Contact Info

Product

Resources

About