Augmenting film and video footage with sensor data

Abstract

Abstract-With the advent of tiny networked devices, MarkWeiser's vision of a world embedded with invisible computers is coming to age. Due to their small size and relative ease of deployment, sensor networks have been utilized by zoologists, seismologists and military personnel. In this paper, we investigate the novel application of sensor networks to the film industry. In particular, we are interested in augmenting film and video footage with sensor data. Unobtrusive sensors are deployed on a film set or in a television studio and on performers. During a filming of a scene, sensor data such as light intensity, color temperature and location are collected and synchronized with each film or video frame. Later, editors, graphics artists and programmers can view this data in synchronization with film and video playback. For example, such data can help define a new level of seamless integration between computer graphics and real world photography. A real-time version of our system would allow sensor data to trigger camera movement and cue special effects. In this paper, we discuss the design and implementation of the first part of our embedded film set environment, the augmented recording system. Augmented recording is a foundational component for the UCLA Hypermedia Studio's research into the use of sensor networks in film and video production. In addition, we have evaluated our system in a television studio.

SENSeTREAM

“…The approaches can be classified into two types: independent and embedded data transfer. Norman et al [13] developed augmented film and video footage with sensor data for the film industry. They store timestamped sensor data and video data in a database, and retrieve the combined data by achieving time synchronization between the data.…”

Embedding expression: Pervasive computing architecture for art and entertainment

“…In UCLA's Advanced Technology for Cinematography (ATC) project [12], we are researching systems to assist the onset production of live action films as a way of motivating Percom research sensitive to filmmaking processes and to uncover these elemental roles. ATC initially focuses on distributed observation and decision support [51] and methods to record the conditions in which shots are created [72] and assist with recreating them.…”

SEVA

“…Automatic annotation by the learning and vision-based techniques is error prone and has high computational requirements. Sensor Annotation of Multimedia: Several systems annotate images, videos, and audio with sensor data such as GPS readings, light readings, temperature readings [1,4,7,14,29,36,37], and use these sensor data to help media retrieval. Many of these systems automatically tag images with time and GPS coordinates of where the image was taken, and then infer other information about the image later.…”