Coordinate-free calibration of an acoustically driven camera pointing system

Abstract

We present a camera pointing system controlled by real-time calculations of sound source locations from a microphone array. Traditional audio localization techniques require explicit estimates of the spatial coordinates for each microphone in the array. In addition, positional information for the camera is needed to use such techniques to drive a camera pointing system. Sometimes this positioning can be done by hand, but for large aperture microphone arrays with many elements this is impractical. We show that in this setting, where elements are placed in an ad-hoc manner, explicitly learning the microphone positions is an unnecessary step. We give a calibration method whose focus is learning the mapping from time delays between pairs of microphones to the associated pan and tilt a PTZ-camera should be given to point at. This curtails the need to explicitly learn the microphone and camera positions. We use this method to calibrate a real-time camera pointing system used by the UCSD interactive display.

Detecting, tracking and interacting with people in a public space

“…Our localization method is based upon learning a regressor L : ∆ → (θ, φ). It has previously been shown that restricting L to be globally linear or quadratic can lead to reasonable accuracies for the camera pointing problem [8]. We extend this work, by using a space partitioning tree wherein linear mappings in the partition cells are used.…”

Alignment of binocular-binaural data using a moving audio-visual target

“…Then the spatial alignment of the two modalities is straightforward and consists in finding the relationship between the microphone-centred and visual-centred coordinate frames such that the two types of sensors refer to the same metric representation. While these methods are well suited for smart-room environments and near-field interaction such as smart kiosks, where a large number of cameras and microphones can be deployed [11], [12], they are not practical in the case of a binaural-binocular active robot head. Indeed, they cannot be applied to just two microphones, they assume stationary sensors and require multiple and perfectly synchronized sound sources.…”

Approaches and databases for online calibration of binaural sound localization for robotic heads

“…In the second phase, this database is used as training data to some machine learning procedure to estimate the mapping between audio features and spatial location. While a number of studies [3], [4], [5] have shown that these approaches work well for robots that operate in dynamic environments, it is unclear that the models learned in one condition will generalize to new acoustic environments. Thus, it is important for robotic systems to adapt to changing acoustic conditions by continually collecting new statistics of the auditory-spatial map and adjusting location estimates accordingly.…”

“…Ben-Reuven and Singer [2] formulate one-dimensional sound localization by discretizing the space of possible source locations and reducing the problem to multi-category classification. Ettinger and Freund [5] use regression rather than classification to learn the mapping between sound features (in this case temporal delays between pairs of microphones in a microphone array) and servo positions that will drive the camera to look at the center of a sound source given a database of training samples. Saxena and Ng [3] compute the incident angle of audio using a single microphone equipped with an artificial pinna by exploiting the pinna's direction specific modulation of the sound signal.…”

“…In recent years there has been an emerging literature focused on the application of machine learning approaches to sound localization [3], [2], [4], [5]. Ben-Reuven and Singer [2] formulate one-dimensional sound localization by discretizing the space of possible source locations and reducing the problem to multi-category classification.…”