Crowdsourced time-sync video tagging using semantic association graph

Yang, Wenmian; Ruan, Na; Gao, Wenyuan; Wang, Kun; Ran, Wensheng; Jia, Weijia

doi:10.1109/icme.2017.8019364

Cited by 22 publications

(11 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Intuitively, TSCs including video tags are usually within hot topics and impact on the trend of their follow-up TSCs. On the contrary, the noises usually neither have similar semantic relationships with other TSCs over a period nor influence other TSCs [58]. Moreover, we find that the density of TSCs (number of TSCs per unit time) affects how users communicate.…”

Section: Introductionmentioning

confidence: 80%

“…However, their approach does not consider the semantic association between TSCs, so that some of the video content-independent noises cannot be processed. In summary, TSCs have some features distinguished from the common comments [32,58], which make the above methods not very effective in the TSCs: (1) Semantic relevance. Abundant video semantic information is contained that describes both local and global video contents by selecting the time interval of the timestamp.…”

Section: Introductionmentioning

confidence: 99%

“…Particularly, this paper is an extended version of [58]. In this extended version, we propose a novel topic center-based cluster algorithm at first, which is more suitable for high-density TSCs.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Time-Sync Video Tag Extraction Using Semantic Association Graph

Yang

Wang

Ruan

et al. 2019

ACM Trans. Knowl. Discov. Data

Self Cite

View full text Add to dashboard Cite

Time-sync comments reveal a new way of extracting the online video tags. However, such time-sync comments have lots of noises due to users' diverse comments, introducing great challenges for accurate and fast video tag extractions. In this paper, we propose an unsupervised video tag extraction algorithm named Semantic Weight-Inverse Document Frequency (SW-IDF). Specifically, we first generate corresponding semantic association graph (SAG) using semantic similarities and timestamps of the time-sync comments. Second, we propose two graph cluster algorithms, i.e., dialogue-based algorithm and topic center-based algorithm, to deal with the videos with different density of comments. Third, we design a graph iteration algorithm to assign the weight to each comment based on the degrees of the clustered subgraphs, which can differentiate the meaningful comments from the noises. Finally, we gain the weight of each word by combining Semantic Weight (SW) and Inverse Document Frequency (IDF). In this way, the video tags are extracted automatically in an unsupervised way. Extensive experiments have shown that SW-IDF (dialogue-based algorithm) achieves 0.4210 F1-score and 0.4932 MAP (Mean Average Precision) in high-density comments, 0.4267 F1-score and 0.3623 MAP in low-density comments; while SW-IDF (topic center-based algorithm) achieves 0.4444 F1-score and 0.5122 MAP in high-density comments, 0.4207 F1-score and 0.3522 MAP in low-density comments. It has a better performance than the state-of-the-art unsupervised algorithms in both F1-score and MAP.

show abstract

Section: Introductionmentioning

confidence: 80%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Time-Sync Video Tag Extraction Using Semantic Association Graph

Yang

Wang

Ruan

et al. 2019

ACM Trans. Knowl. Discov. Data

Self Cite

View full text Add to dashboard Cite

show abstract

“…The higher the neighbor similarity and the lower the keyframe similarity, the less likely the TSC is a spoiler. Considering the highnoise property, we implement Interactive Variance Attention (IVA) mechanism in the framework to effectively reduce the impact of noise due to the low semantic similarities between the noise and its surrounding TSCs [34]. Finally, we obtain the likelihood of the spoilers based on the difference between the neighbor and keyframe similarity.…”

Section: Keyframes Commentsmentioning

confidence: 99%

“…Wu et al [29] first formally introduce the TSC, and propose a preliminary method to extract video tags. Yang et al [34] systematically collate the features of TSCs and introduces a graphbased algorithm to eliminate the impact of noise prominently. Then further usages of TSCs data are proposed like extracting highlight shots [24,30], labeling important segments [19], detecting events [16], generating temporal descriptions of videos [31], and video recommendation [5,23,32].…”

Section: Related Workmentioning

confidence: 99%

Interactive Variance Attention based Online Spoiler Detection for Time-Sync Comments

Yang

Jia²,

Gao

et al. 2019

Proceedings of the 28th ACM International Conference on Information and Knowledge Management

Self Cite

View full text Add to dashboard Cite

Nowadays, time-sync comment (TSC), a new form of interactive comments, has become increasingly popular on Chinese video websites. By posting TSCs, people can easily express their feelings and exchange their opinions with others when watching online videos. However, some spoilers appear among the TSCs. These spoilers reveal crucial plots in videos that ruin people's surprise when they first watch the video. In this paper, we proposed a novel Similarity-Based Network with Interactive Variance Attention (SBN-IVA) to classify comments as spoilers or not. In this framework, we firstly extract textual features of TSCs through the word-level attentive encoder. We design Similarity-Based Network (SBN) to acquire neighbor and keyframe similarity according to semantic similarity and timestamps of TSCs. Then, we implement Interactive Variance Attention (IVA) to eliminate the impact of noise comments. Finally, we obtain the likelihood of spoiler based on the difference between the neighbor and keyframe similarity. Experiments show SBN-IVA is on average 11.2% higher than the state-of-the-art method on F1-score in baselines.

show abstract