Smart Director: An Event-Driven Directing System for Live Broadcasting

Abstract

Live video broadcasting normally requires a multitude of skills and expertise with domain knowledge to enable multi-camera productions. As the number of cameras keeps increasing, directing a live sports broadcast has now become more complicated and challenging than ever before. The broadcast directors need to be much more concentrated, responsive, and knowledgeable, during the production. To relieve the directors from their intensive efforts, we develop an innovative automated sports broadcast directing system, called Smart Director, which aims at mimicking the typical human-in-the-loop broadcasting process to automatically create near-professional broadcasting programs in real-time by using a set of advanced multi-view video analysis algorithms. Inspired by the so-called “three-event” construction of sports broadcast [ 14 ], we build our system with an event-driven pipeline consisting of three consecutive novel components: (1) the Multi-View Event Localization to detect events by modeling multi-view correlations, (2) the Multi-View Highlight Detection to rank camera views by the visual importance for view selection, and (3) the Auto-Broadcasting Scheduler to control the production of broadcasting videos. To our best knowledge, our system is the first end-to-end automated directing system for multi-camera sports broadcasting, completely driven by the semantic understanding of sports events. It is also the first system to solve the novel problem of multi-view joint event detection by cross-view relation modeling. We conduct both objective and subjective evaluations on a real-world multi-camera soccer dataset, which demonstrate the quality of our auto-generated videos is comparable to that of the human-directed videos. Thanks to its faster response, our system is able to capture more fast-passing and short-duration events which are usually missed by human directors.

Ball Trajectory Inference from Multi-Agent Sports Contexts Using Set Transformer and Hierarchical Bi-LSTM

“…Automatic sports broadcasting inevitably involves tracking the ball or players to zoom in on the region of interest from the whole pitch [6,38,53]. Figure 5 demonstrates two snapshots obtained by setting the estimated ball position by our model to the center of the snapshots.…”

Research on the Development of Sports in the Age of Artificial Intelligence

“…Traditional sports have significant industry pain points, such as inadequate offline sports experience and information asymmetry between venues and athletes. Through internet technology, we can broadcast live sports events to remote audiences or sports enthusiasts [11] . The teaching content of good coaches or sports teachers can be presented to more people or students in remote areas [12] .…”