Confidence Building Among Correlated Streams in Multimedia Surveillance Systems

Abstract

Abstract. Multimedia surveillance systems utilize multiple correlated media streams, each of which has a different confidence level in accomplishing various surveillance tasks. For example, the system designer may have a higher confidence in the video stream compared to the audio stream for detecting humans running events. The confidence level of streams is usually precomputed based on their past accuracy. This traditional approach is cumbersome especially when we add a new stream in the system without the knowledge of its past history. This paper proposes a novel method which dynamically computes the confidence level of new streams based on their agreement/disagreement with the already trusted streams. The preliminary experimental results show the utility of our method.

Multimodal fusion for multimedia analysis: a survey

“…Recently, Atrey et al [10] have also presented a method to dynamically compute the confidence level of a media stream based on its agreement coefficient with a trusted stream. The trusted stream is the one that has the confidence level above a certain threshold.…”

“…Tavakoli et al [127], Atrey et al [10] have also been performed towards the dynamic computation of the confidence levels of different modalities. Tavakoli et al [127] have used spatial and temporal information in clusters in order to determine a confidence level of sensors.…”

Learning Multi-Sensor Confidence using Difference of Opinions

“…The authors used the past accuracy of the sensors to compute the confidence but did not address the evolution of such confidence over time. The work in [6] and [1] proposed a dynamic method to compute the confidence of individual sensors. The model presented in [6] considers the performance of individual sensors over a period of time compared to other sensors to compute the confidence of each sensor.…”

“…The core motivation of using multiple sensors is to obtain a better perception of the observation environment and the activities happening in the environment. Usually different sensors have different confidence levels in performing various observation tasks [1]. For example, a video stream may have higher confidence than an audio stream in identifying a person.…”

“…Although, their method dynamically measured the confidence in each sensor, it is not clear how they measure the similarity of performance among sensors. A different approach has been adopted in [1] where the authors initially consider a set of trusted media streams to compute the confidence of other non-trusted streams. They used the agreement/disagreement among the trusted and non-trusted media streams and based on this association, compute the evolving confidence of the nontrusted streams using a Bayesian formulation, while the confidence of the already trusted streams remains static.…”