Dynamic Context Capture and Distributed Video Arrays for Intelligent Spaces

Trivedi, Mohan M.; Huang, K.S.; Mikić, Ivana Medvedec

doi:10.1109/tsmca.2004.838480

Cited by 76 publications

(36 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…can be applied to captured audio-visual signals to extract higher-level semantic information, such as identification and location in real time. A system that combines face and audio based identification along with motion detection, person tracking and audio based localization has been proposed in the literature [21]. Such a system applies state-of-the-art methods to process results from each individual modality and uses particle filtering to fuse both modalities for providing robust identification and localization.…”

Section: Simultaneous Identification and Localizationmentioning

confidence: 99%

A System for Multimodal Context-Awareness

Galatas¹,

Makedon²

2013

IJACSA

View full text Add to dashboard Cite

Abstract-in this paper we present the improvement of our novel localization system, by introducing radio-frequency identification (RFID) which adds person identification capabilities and increases multi-person localization robustness. Our system aims at achieving multi-modal context-awareness in an assistive, ambient intelligence environment. The unintrusive devices used are RFID and 3-D audio-visual information from 2 Kinect sensors deployed at various locations of a simulated apartment to continuously track and identify its occupants, thus enabling activity monitoring. More specifically, we use skeletal tracking conducted on the depth images and sound source localization conducted on the audio signals captured by the Kinect sensors to accurately localize and track multiple people. RFID information is used mainly for identification purposes but also for rough location estimation, enabling mapping of the location information from the Kinect sensors to the identification events of the RFID. Our system was evaluated in a real world scenario and attained promising results exhibiting high accuracy, therefore showing the great prospect of using the RFID and Kinect sensors jointly to solve the simultaneous identification and localization problem.

show abstract

Section: Simultaneous Identification and Localizationmentioning

confidence: 99%

A System for Multimodal Context-Awareness

Galatas¹,

Makedon²

2013

IJACSA

View full text Add to dashboard Cite

show abstract

“…In our ongoing research on driver-assistance systems [27], [28] and audiovisual scene understanding [29], [30], emotion recognition using multimodality will certainly help us improve the interface. With this motivation, we have put in significant effort on collection of audiovisual affect database in a challenging ambient of car settings [31].…”

Section: B Audio-visual Affect Database (Cvrrcar-avdb)mentioning

confidence: 99%

Speech Emotion Analysis: Exploring the Role of Context

Tawari

Trivedi

2010

IEEE Trans. Multimedia

View full text Add to dashboard Cite

Abstract-Automated analysis of human affective behavior has attracted increasing attention in recent years. With the research shift toward spontaneous behavior, many challenges have come to surface ranging from database collection strategies to the use of new feature sets (e.g., lexical cues apart from prosodic features). Use of contextual information, however, is rarely addressed in the field of affect expression recognition, yet it is evident that affect recognition by human is largely influenced by the context information. Our contribution in this paper is threefold. First, we introduce a novel set of features based on cepstrum analysis of pitch and intensity contours. We evaluate the usefulness of these features on two different databases: Berlin Database of emotional speech (EMO-DB) and locally collected audiovisual database in car settings (CVRRCar-AVDB). The overall recognition accuracy achieved for seven emotions in the EMO-DB database is over 84% and over 87% for three emotion classes in CVRRCar-AVDB. This is based on tenfold stratified cross validation. Second, we introduce the collection of a new audiovisual database in an automobile setting (CVRRCar-AVDB). In this current study, we only use the audio channel of the database. Third, we systematically analyze the effects of different contexts on two different databases. We present context analysis of subject and text based on speaker/text-dependent/-independent analysis on EMO-DB. Furthermore, we perform context analysis based on gender information on EMO-DB and CVRRCar-AVDB. The results based on these analyses are promising.

show abstract

“…The method of video-only tracking [4], [5] is generally reliable and accurate when the targets are in the camera field of view, but limitations are introduced when the targets are occluded by other speakers, when they disappear from the camera field of view, or the appearance of the targets or illumination is changed [3], [6]. Audio tracking [7], [8], [9] is not restricted by these limitations, however, audio data is intermittent over time and may be corrupted by background noise and room reverberations, which may introduce non-negligible tracking errors.…”

Section: Introductionmentioning

confidence: 99%

Audio Assisted Robust Visual Tracking With Adaptive Particle Filtering

Kılıç

Barnard

Wang

et al. 2015

IEEE Trans. Multimedia

View full text Add to dashboard Cite

Abstract-The problem of tracking multiple moving speakers in indoor environments has recently received much attention. Earlier techniques were based purely on vision, but the theoretical and algorithmic advances and a constant growth in speed of processing have led to the emergence of techniques which allow the fusion of audio and visual data. The fusion of multi-modal information has been shown to be instrumental in improving tracking performance, as well as robustness in the case of challenging situations like occlusions (by the limited field of view of cameras or by other speakers). However, data fusion algorithms often suffer from noise corrupting the sensor measurements which cause non-negligible detection errors. Here, a novel approach to combining audio and visual data is proposed. In our framework, we employ audio data as an aid to particle filter (PF) based visual tracking, by using the direction of arrival angles of the audio sources to reshape the typical Gaussian noise distribution of particles in the propagation step and to weight the observation model in the measurement step. This approach is further improved by solving a typical problem associated with the PF. It has been observed that the efficiency and accuracy of the PF usually depend on the number of particles and noise variance used in the estimation and propagation functions for re-allocating these particles at each iteration. Both of these parameters are specified beforehand and are kept fixed in the regular implementation of the PF which makes the tracker unstable in practice. To address these problems, we design an algorithm which adapts both the number of particles and noise variance based on tracking error and the area occupied by the particles in the image. Experiments on the AV 16.3 dataset show the advantage of our proposed methods over the baseline PF method and an existing adaptive PF algorithm for tracking occluded speakers with a significantly reduced number of particles.Index Terms-Audio-visual speaker tracking, particle filter, adaptive particle filter.

show abstract

Dynamic Context Capture and Distributed Video Arrays for Intelligent Spaces

Cited by 76 publications

References 36 publications

A System for Multimodal Context-Awareness

A System for Multimodal Context-Awareness

Speech Emotion Analysis: Exploring the Role of Context

Audio Assisted Robust Visual Tracking With Adaptive Particle Filtering

Contact Info

Product

Resources

About