Finding meaning on YouTube: Tag recommendation and category discovery

Abstract

We present a system that automatically recommends tags for YouTube videos solely based on their audiovisual content. We also propose a novel framework for unsupervised discovery of video categories that exploits knowledge mined from the World-Wide Web text documents/searches. First, video content to tag association is learned by training classifiers that map audiovisual content-based features from millions of videos on YouTube.com to existing uploadersupplied tags for these videos. When a new video is uploaded, the labels provided by these classifiers are used to automatically suggest tags deemed relevant to the video. Our system has learned a vocabulary of over 20,000 tags. Secondly, we mined large volumes of Web pages and search queries to discover a set of possible text entity categories and a set of associated is-A relationships that map individual text entities to categories. Finally, we apply these is-A relationships mined from web text on the tags learned from audiovisual content of videos to automatically synthesize a reliable set of categories most relevant to videos -along with a mechanism to predict these categories for new uploads. We then present rigorous rating studies that establish that: (a) the average relevance of tags automatically recommended by our system matches the average relevance of the uploader-supplied tags at the same or better coverage and (b) the average precision@K of video categories discovered by our system is 70% with K=5.

YouTubeEvent: On large-scale video event classification

“…The idea of automatic category discovery in this work shares some spirits with [24]. In [24], a set of entity categories are discovered by mining web pages and search queries.…”

Feature extraction and statistical analysis of videos for cinemetric applications

“…64) and resize the frame to 10%. Resizing the frame and keeping only a small percentage helps to limit the computations, and does not affect the results, as the color information is not highly affected by image quality [5] [11]. Figure 1 present the original image and the quantized and resized copy.…”

“…Having acquired information for each frame, it is relatively easy to translate it at the shot level, by employing the average face ratio from all the frames of the shot. In order to avoid having the shot results affected by a possibly very large (or a very small) BB, we first sort the vector that contains all the BB sizes for all the frames, and then select the median value, as in [1] [11]. Having extracted the face-frame ratios, we could use this information to extract the shot-type as in [7].…”

On the Challenges and Opportunities in Visualization for Machine Learning and Knowledge Extraction: A Research Agenda

“…Clustering methods are often subject to other systems, for example to reduce the possibility of recommender systems (e.g. Tag-recommender on Youtube videos [10]); for example clustering of large high-dimensional gene expression data sets has widespread application in -omics [11]. Unfortunately, the underlying structure of these natural data sets is often fuzzy, and the computational identification of data clusters generally requires (human) expert knowledge about cluster number and geometry.…”