Time-Sync Video Tag Extraction Using Semantic Association Graph

Abstract

Time-sync comments reveal a new way of extracting the online video tags. However, such time-sync comments have lots of noises due to users' diverse comments, introducing great challenges for accurate and fast video tag extractions. In this paper, we propose an unsupervised video tag extraction algorithm named Semantic Weight-Inverse Document Frequency (SW-IDF). Specifically, we first generate corresponding semantic association graph (SAG) using semantic similarities and timestamps of the time-sync comments. Second, we propose two graph cluster algorithms, i.e., dialogue-based algorithm and topic center-based algorithm, to deal with the videos with different density of comments. Third, we design a graph iteration algorithm to assign the weight to each comment based on the degrees of the clustered subgraphs, which can differentiate the meaningful comments from the noises. Finally, we gain the weight of each word by combining Semantic Weight (SW) and Inverse Document Frequency (IDF). In this way, the video tags are extracted automatically in an unsupervised way. Extensive experiments have shown that SW-IDF (dialogue-based algorithm) achieves 0.4210 F1-score and 0.4932 MAP (Mean Average Precision) in high-density comments, 0.4267 F1-score and 0.3623 MAP in low-density comments; while SW-IDF (topic center-based algorithm) achieves 0.4444 F1-score and 0.5122 MAP in high-density comments, 0.4207 F1-score and 0.3522 MAP in low-density comments. It has a better performance than the state-of-the-art unsupervised algorithms in both F1-score and MAP.

Interactive Variance Attention based Online Spoiler Detection for Time-Sync Comments

“…The younger brother is a wanted criminal, and the delivery guy know his identity Another important property of TSCs is interactive. The topic of the latter posted TSCs usually depends on the former, which is similar to the herding mentality effect [10,32,35] in social science. In Fig.…”

Interpretable video tag recommendation with multimedia deep learning framework

“…Recently, given the increasing popularity of video-sharing platforms, video tag recommender systems that recommend high-quality tags to online videos have been developed for YouTube (Parra et al , 2018; Toderici et al , 2010; Wang et al , 2017), Instagram (Li et al , 2019; Wei et al , 2019), Niconico (Sakaji et al , 2016) and Bilibili (Lv et al , 2016; Yang et al , 2019). Most video tag recommender systems utilize the sheer volume of user-contributed videos and comments to enhance their recommendation performance (Lau et al , 2018).…”

“…A tag recommender system at this stage utilizes content solely submitted by the video producer, such as text description, audio and video frame (Li et al , 2019). After releasing the video, the comments and time-sync comments generated by video viewers are used as additional data sources for tag recommender systems that seek to refine the initial tags (Sakaji et al , 2016; Yang et al , 2019).…”

“…Since less than half of the Chinese netizens hold a high school diploma or above (China Internet Network Information Center, 2019) and most Bilibili users are young people, guidance for tagging videos is needed. Although extant studies, such as Lv et al (2016) and Yang et al (2019), construct video tag recommender systems for Bilibili, these systems are only suitable for refining the tags after releasing the video because they employ time-sync comments. Moreover, these previously designed systems fail to provide human-understandable explanations.…”

“…To aid the tagging process, artificial intelligence-enabled (AI-enabled) video tag recommender systems that recommend accurate tags for videos have been created for popular video-sharing platforms, including YouTube (Parra et al , 2018; Toderici et al , 2010; Wang et al , 2017), Instagram (Li et al , 2019; Wei et al , 2019) and Bilibili (Lv et al , 2016; Yang et al , 2019). These video tag recommender systems incorporate various AI techniques and utilize online multimedia big data to enhance model performances.…”