Augmenting Live Broadcast Sports with 3D Tracking Information

Abstract

In this article we describe a family of technologies designed to enhance live television sports broadcasts. Our algorithms provide a means for tracking the game object (the ball in baseball, the puck in hockey); instrumenting broadcast cameras and creating informative graphical visualizations regarding the object's trajectory. The graphics are embedded in the live image in real time in a photo realistic manner. The demands of live network broadcasting require that many of our effects operate on live video streams at frame-rate (60 Hz) with latencies of less than a second. Some effects can be composed off-line, but must be Preprint submitted to IEEE MultiMedia Magazine complete in a matter of seconds. The final image sequence must also be compelling, appealing and informative to the critical human eye. Components of our approach include:(1) Instrumention and calibration of broadcast cameras so that the geometry of their field of view is known at all times, (2) Tracking items of interest such as balls and pucks and in some cases participants (e.g., cars in auto racing), (3) Creation and transformation of graphical enhancements such as illustrative object highlights (e.g., a glowing puck trail) or advertisements to the same field of view as the broadcast cameras and (4) Techniques for combining these elements into informative on-air images. We describe each of these technologies and illustrate their use in network broadcast of hockey, baseball, football and auto racing. We focus on vision based tracking of game objects.

The multi‐iterative closest point tracker: An online algorithm for tracking multiple interacting targets

“…We were able to integrate the tracks with video of the game using algorithms developed by Sportvision (Figure 8) (Cavallaro, Hybinette, White, & Balch, 2011). In this video, the location of each player is indicated with a rectangle, and colored trails illustrate their tracked positions.…”

“…In this video, the location of each player is indicated with a rectangle, and colored trails illustrate their tracked positions. For more details on this application, see (Cavallaro et al, 2011).…”

EAR: Enhanced Augmented Reality System for Sports Entertainment Applications

“…The AR and Computer Vision In the AR applications, virtual entities can change the way the real world is perceived by the users' by adding, removing, or changing few aspects of the real world [7]. For example, a simulated image or video of a city may contain useful information about buildings, parking-lots, and restaurants [2].…”

“…The proposed work facilitates viewers by a touch-screen stimulated mechanism. Rick et al [15] present a system for augmenting live broadcast sports with 3-D information by inserting a graphical element into a live soccer scene. Jungong et al [16] demonstrate an AR system on volley ball and tennis during live sports.…”

On domain-specific decision support systems for e-sports strategy games

“…These systems include databases of meta-game information, augmented reality systems (Cavallaro et al 2011), object recognition systems (Mahmood et al 2012) and camera positioning systems (Yus et al 2011) The development of Commentator Support Systems has previously been driven by television broadcasters seeking to provide effective and unique viewing experiences (Miller & Washington 2012) (Denia et al 2011), but recently, this focus has shifted towards the use of internetbased multimedia developments so end-users can compile their own information sources (Seo et al 2012).…”