Collaborative 3D Target Tracking in Distributed Smart Camera Networks for Wide-Area Surveillance

Abstract

With the evolution and fusion of wireless sensor network and embedded camera technologies, distributed smart camera networks have emerged as a new class of systems for wide-area surveillance applications. Wireless networks, however, introduce a number of constraints to the system that need to be considered, notably the communication bandwidth constraints. Existing approaches for target tracking using a camera network typically utilize target handover mechanisms between cameras, or combine results from 2D trackers in each camera into 3D target estimation. Such approaches suffer from scale selection, target rotation, and occlusion, drawbacks typically associated with 2D tracking. In this paper, we present an approach for tracking multiple targets directly in 3D space using a network of smart cameras. The approach employs multi-view histograms to characterize targets in 3D space using color and texture as the visual features. The visual features from each camera along with the target models are used in a probabilistic tracker to estimate the target state. We introduce four variations of our base tracker that incur different computational and communication costs on each node and result in different tracking accuracy. We demonstrate the effectiveness of our proposed trackers by comparing their performance to a 3D tracker that fuses the results of independent 2D trackers. We also present performance analysis of the base tracker along Quality-of-Service (QoS) and Quality-of-Information (QoI) metrics, and study QoS vs. QoI trade-offs between the proposed tracker variations. Finally, we demonstrate our tracker in a real-life scenario using a camera network deployed in a building.

Efficient network clustering for traffic reduction in embedded smart camera networks

“…Coordination in multi-camera networks has been done in [12,25,32] using hand off of relevant data that could be used to infer a knowledge previously detected in neighbor cameras. The method proposed in [31] uses video data compression to reduce communication latency and increase the bandwidth in a distributed smart camera network.…”

Sensor Fusion with Asynchronous Decentralized Processing for 3D Target Tracking with a Wireless Camera Network

“…However, much less attention has been given to the often-occurring scenario in which targets move in 3D space while taking into account network asynchrony and delays in image processing. For example, in [ 22 ], multi-view histograms were employed to characterize targets in 3D space using color and texture as the visual features; the implementation was based on particle filtering. In [ 23 ], a deep network model was trained with an RGB-D dataset, and then the 3D structure of the target was reconstructed from the corresponding RGB image to predict depth information.…”

On Optimal Multi-Sensor Network Configuration for 3D Registration

“…Synergy among several heterogeneous sensors can provide more precise results, especially when there are mobile networks of sensors involved, for example, visual sensors mounted on a mobile robot [1]. The heterogenous sensor networks require fusion strategies for maximizing information content [2][3][4][5][6][7] and optimal configuration of sensor networks are thus very important.…”