An Audio-Visual Database For Evaluating Person Tracking Algorithms

Abstract

This paper presents an audio-visual database that can be used as a reference database for testing and evaluation of video, audio or joint audio-visual person tracking algorithms, as well as speaker localization methods. Additional possible uses include the testing of face detection and pose estimation algorithms. A number of different scenes are included in the database, ranging from simple to complex scenes that can challenge existing algorithms. They include different subjects, with appearances that can cause problems to video tracking algorithms, (e.g. facial features such as beards, glasses, etc.), optimal and artificially created sub-optimal lighting conditions, subject movement based on simple as well as random motion trajectories, different distances from the camera/microphones and occlusion. The database incorporates ground truth data (3-D position in time) originating from a commercially available 4-camera infrared (IR) tracking system. Examples of how the database can be used to evaluate video and audio tracking algorithms are also provided.

A monocular system for person tracking: Implementation and testing

“…The ideal case involves direct comparison of the output of a tracking system against reference or ground truth data. Video sequences with ground truth were available for the purpose of this study [44]. The sequences were obtained in the Virtual Studio of the Technical University of Ilmenau in Germany, as part of the CARROUSO ("Creating, Assessing and Rendering in Real Time of High Quality AudioVisual Environments in MPEG-4 Context" [45]) European Research Project.…”

“…It should be noted, though, that the frame rate can substantially increase (12-15 frames/sec) at the expense of accuracy if the frames are sub-sampled prior to processing or certain internal parameters of the detection algorithms are relaxed. Test video sequences include the CARROUSO project video sequences [44]. In Figures 6, 7 and 8, the results of automatic face detection and 2-D tracking for three of the CARROUSO project video sequences are illustrated.…”

An MPEG-7 Based Description Scheme for Video Analysis Using Anthropocentric Video Content Descriptors

“…The proposed classes are an amalgam of low-level features, like histograms, FFTs etc, and high semantic entities, like scene theme, person recognition, emotions, sounds qualification, etc. Nowadays researchers are very interested in extracting high semantic features [9], [10], [11]. Having this in mind, we believe that the proposed template can give the genre of a new digital video processing approach.…”

Simultaneous Localization of Multiple Sound Sources using Blind Adaptive MIMO Filtering

“…These data are made available as part of an audiovisual database [18]. This database also includes reference data of the speaker positions measured using infrared sensors.…”

“…1 (a) and (b), we see that (18) and (19) can in fact directly be considered as the generalization of the AED condition (2) to multiple sources. Using a proper coefficient initialization, (18) is the corresponding equation to estimate the TDOA of source 1, while (19) gives the TDOA of source 2. Moreover, since the coefficient initialization, described in Sect.…”