Extracting Hierarchical Structure of Web Video Groups Based on Sentiment-Aware Signed Network Analysis

Abstract

Sentiment in multimedia contents has an influence on their topics, since multimedia contents are tools for social media users to convey their sentiment. Performance of applications such as retrieval and recommendation will be improved if sentiment in multimedia contents can be estimated; however, there have been few works in which such applications were realized by utilizing sentiment analysis. In this paper, a novel method for extracting the hierarchical structure of Web video groups based on sentiment-aware signed network analysis is presented to realize Web video retrieval. First, the proposed method estimates latent links between Web videos by using multimodalfeatures of contents and sentiment features obtained from texts attached to Web videos. Thus, our method enables construction of a signed network that reflects not only similarities but also positive and negative relations between topics of Web videos. Moreover, an algorithm to optimize a modularity-based measure, which can adaptively adjust the balance between positive and negative edges, was newly developed. This algorithm detects Web video groups with similar topics at multiple abstraction levels; thus, successful extraction of the hierarchical structure becomes feasible. By providing the hierarchical structure, users can obtain an overview of many Web videos and it becomes feasible to successfully retrieve the desired Web videos. Results of experiments using a new benchmark dataset, YouTube-8M, validate the contributions of this paper, i.e., 1) the first attempt to utilize sentiment analysis for Web video grouping and 2) a novel algorithm for analyzing a weighted signed network derived from sentiment and multimodal features

Community Detection Based on Local Information and Dynamic Expansion

“…At the same time, Jiang [39] applied the Signed Stochastic Block-Model into community detection; Harakawa et al [40] divided the web video by constructing a weighted signed network and maximizing the local modularity; Li et al [41] proposed a new algorithm based on the improved function of the modularity and the density, while studied the performance of the proposed algorithm under various parameters.…”

Sentiment-aware personalized tweet recommendation through multimodal FFM

“…However, most of the existing methods for sentiment analysis for multimedia contents such as images, audio and videos are fundamental methods to predict sentiment in multimedia contents [4,6,17,19,29,47]. In contrast, there has been a pioneer work [14] that used sentiment features for successfully realizing an application, i.e., Web video retrieval. However, personalized tweet recommendation using sentiment features has not been proposed to the best of our knowledge.…”

Tracking topic evolution via salient keyword matching with consideration of semantic broadness for Web video discovery

“…In this paper, the hierarchical structure denotes the property of Web video groups being divided into sub-groups. Several methods have been proposed for extracting the hierarchical structure [20,21,23,24,45,46,48,51] and for tracking topic evolution of Web video groups [6,8,10,47,55]; however, our novel method enables simultaneous realization of extraction and tracking of topic evolution in the hierarchical way. For each time stamp, our novel method extracts the hierarchical structure and salient keywords that represent contents of each Web video group on the basis of network analysis [4] using multimodal features, i.e., features of video contents and metadata.…”

“…Taskiran et al [51] proposed "a similarity pyramid" to browse video shots at various levels of detail by using visual features. To overcome the performance limitation of these methods using a single modality, we proposed methods utilizing multimodal features [20,21,23,24]. These methods enable users to retrieve Web videos containing topics with desired semantic broadness by hierarchically providing Web video groups.…”

“…We solved "Problem (i)" by providing the hierarchical structure of Web video groups. In previous works [20,21,23,24], it was proven that the hierarchical structure can navigate users to the desired Web videos even if users cannot input suitable queries. We solved "Problem (ii)" by the P-1 P-2 P-3 P-4 R1 R2-1 R2-2 R2-3 R2-4 R3 1. proposed tracking method that seeks similar Web video groups over time.…”