Automatic Content Curation System for Multiple Live Sport Video Streams

Abstract

In this paper, we aim to develop a method to create personalized and high-presence multi-channel contents for a sport game through realtime content curation from various media streams captured/created by spectators. We use the live TV broadcast as a ground truth data and construct a machine learning-based model to automatically conduct curation from multiple videos which spectators captured from different angles and zoom levels. The live TV broadcast of a baseball game has some curation rules which select a specific angle camera for some specific scenes (e.g., a pitcher throwing a ball). As inputs for constructing a model, we use meta data such as image feature data (e.g., a pitcher is on the screen) in each fixed interval of baseball videos and game progress data (e.g., the inning number and the batting order). Output is the camera ID (among multiple cameras of spectators) at each point of time. For evaluation, we targeted Spring-Selection high-school baseball games. As training data, we used image features, game progress data, and the camera position at each point of time in the TV broadcast. We used videos of a baseball game captured from 7 different points in Hanshin Koshien Stadium with handy video cameras and generated sample data set by dividing the videos to fixed interval segments. We divided the sample data set into the training data set and the test data set and evaluated our method through two validation methods: (1) 10-fold crossvalidation method and (2) hold-out methods (e.g., learning first and second innings and testing third inning). As a result, our method predicted the camera switching timings with accuracy (F-measure) of 72.53% on weighted average for the base camera work and 92.1% for the fixed camera work.

Learning Sports Camera Selection From Internet Videos

“…Camera Viewpoint Prediction In single camera systems, previous techniques have been proposed to predict camera angles for PTZ cameras [7] and to generate natural-looking normal field-of-view (NFOV) video from 360 o panoramic views [33,32,22]. In multi-camera systems, camera viewpoint prediction methods select a subset of all available cameras [39,12,35,3,17,18,43,40,26]. In broadcast systems, semi-automatic [16] and fully automatic systems have been developed in practice.…”

Data Streams Curation for Better Machine Learning Functionality and Result to Serve IoT and other Applications: A Survey

“…Challenges that can be found in combining data, its meta-data and a description of the goal needed from the data can lead to the automation of data analytic and this would require using analytic ontology for assisting in data cleaning, suggestions, variable selection and the goal interpretation for the models to use and not only the meta-data learning, which will lead to draw precious explanation for the decision making process based on logical inferences and different machine learning techniques such as Random Forest Algorithm for fast learning on real-time data being ingested (Miller et al, 2017;Fujisawa et al, 2016). Utilizing the predictive data analytic (Functional Regression, Second-Order Autoregressive time-series model, Functional Data Analysis and Principle Data Analysis) can assess the process of drawing theories from the data on motion or the data that rely on the time (e.g., predicting stock prices from income statement data) and this will enhance the process of analytic automation, especially in the case meta-data are including references to other pre-calculated knowledge based on the successful learning examples (Yasumoto et al, 2016), or using wiki technology to make the formation for the knowledge based on the historical data listed in the documents (Tebbakh, 2014).…”

“…When it comes to heavy streaming, more advanced curation techniques to be considered, (Fujisawa et al, 2016), for example, do a video content curation for the cameras on real-time basis in order to create a video consisting of scenes from the different cameras located at different corners at each point of time for sport games is another challenging application for such data curation process, which can be conducted by training a machine learning model on the different images features, the game data and the camera different positions at each point of time, this would require dividing the videos into small segments that facilitate the process of applying machine learning technique. Such learning process implemented by collecting the multiple video streams, which are in turn divided into small segments that are assigned metadata describing them as a curation process, followed by building a time-based estimation model to generate the automatic cameras switching hence curating the games' video content by using the estimated right corner to capture the scenes at the right time based on machine learning and the curated streams.…”

“…Finally, Principle Differential Analysis (Ramsay, 2006) to detect the cases when describing the change in the output functions using ODE. Random Forest Learning Algorithm, which can be considered the fastest and the most suitable algorithm for learning on a real-time basis is considered by the authors in (Fujisawa et al, 2016).…”

Survey of Real-time Processing Technologies of IoT Data Streams

“…Fujisawa et al proposed a video curation system [56] that targets baseball games and automatically creates real-time video content with high values from multiple video streams captured by spectator cameras in different places and at different angles and zoom levels. In their study, assuming that video contents with similar camera switching patterns (i.e., which camera's video is used in the broadcasted content and when) to the TV broadcast have high values, machine learning algorithms are constructed using the camera switching patterns of TV broadcasts as training data.…”