Multimodal identification and localization of users in a smart environment

Abstract

Detecting the location and identity of users is a first step in creating contextaware applications for technologically-endowed environments. We propose a system that makes use of motion detection, person tracking, face identification, feature-based identification, audio-based localization, and audio-based identification modules, fusing information with particle filters to obtain robust localization and identification. The data streams are processed with the help of the generic client-server middleware SmartFlow, resulting in a flexible architecture that runs across different platforms.

Feature-based tracking on a multi-omnidirectional camera dataset

“…The omnidirectional video databases we mention in this section are all acquired with multiple cameras, but none of them use multiple omnidirectional cameras. In meeting corpora, the omnidirectional ceiling camera has been found to be an adequate modality to do the manual annotations, whereas conventional cameras are used to track people [4]. There are also hybrid systems that combine omnidirectional cameras with for instance PTZ cameras [5].…”

“…• ceiling-mounted IP camera with a 5 megapixel sensor with no moving parts 4 . Calibrated camera parameters are distributed along with the dataset.…”

Behavioural Biometrics and Human Identity

“…Dynamic Bayesian networks are popular for probabilistic fusion of biometric evidence, allowing to cope with uncertainties in the input (Maurer and Baker 2007 ) . An important issue is the evaluation of the statistical correlation of the input to such fusion systems ( COGNIRON ; Salah et al 2008 ) . Another dimension of fusion is the architecture, which can be serial or parallel.…”

Sift-Based Camera Localization Using Reference Objects for Application in Multi-Camera Environments and Robotics

“…For this purpose, the identities of the persons in the room have to be determined (see (Salah et al, 2008)) as well as the audio-visual focus of attention has to be estimated (see (Voit and Stiefelhagen, 2010;Schauerte et al, 2009)), e.g. to present personalized information on the display a person is currently looking at.…”