Information assimilation framework for event detection in multimedia surveillance systems

Abstract

Most multimedia surveillance and monitoring systems nowadays utilize multiple types of sensors to detect events of interest as and when they occur in the environment. However, due to the asynchrony among and diversity of sensors, information assimilation -how to combine the information obtained from asynchronous and multifarious sources is an important and challenging research problem. In this paper, we propose a framework for information assimilation that addresses the issues -"when", "what" and "how" to assimilate the information obtained from different media sources in order to detect events in multimedia surveillance systems. The proposed framework adopts a hierarchical probabilistic assimilation approach to detect atomic and compound events. To detect an event, our framework uses not only the media streams available at the current instant but it also utilizes their two important properties -first, accumulated past history of whether they have been providing concurring or contradictory evidences, and -second, the system designer's confidence in them. The experimental results show the utility of the proposed framework.

Confidence Building Among Correlated Streams in Multimedia Surveillance Systems

“…The authors have used the Dempster-Shafer (D-S) theory of evidence to fuse the confidences. In contrast, we propose a model for confidence fusion by using a Bayesian formulation because it is both simple and computationally efficient [4].…”

“…Also, let P Φ (t) be the 'fused probability' of occurrence of the event at time t based on a subset Φ ∈ P(M n ) of media streams. The 'fused probability' is the overall probability of occurrence of the event based on a group of media streams [4]. M3.…”

“…These probabilities represent decisions about the events. Exactly same probabilities would imply full agreement (γ ik = 1) whereas totally dissimilar probabilities would mean that the two streams fully contradict each other (γ ik = −1) [4]. The agreement coefficient between two sources M i−1 and M i is modelled as:…”

“…First, we assume that the system designer's confidence level in each of the media streams is more than 0.5 (Refer to assumption A2, in section 3). Second, although the media streams are correlated in their decisions; we assume that they are mutually independent in terms of their confidence levels [4]. For n number of media streams, the overall confidence is iteratively computed.…”

“…As the different streams have different confidence levels associated for accomplishing different tasks, it is important to utilize the confidence information of streams in their assimilation by appropriately assigning the weights to them [4]. The confidence in a stream is related to its accuracy.…”