The Effect of Pixel-Level Fusion on Object Tracking in Multi-Sensor Surveillance Video

Cvejic, Nedeljko; Nikolov, SG; Knowles, Henry D.; Loza, Artur; Achim, Alin; Bull, David; Canagarajah, C N

doi:10.1109/cvpr.2007.383433

Cited by 56 publications

(25 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Future work on the proposed PF tracking approach will be focussed on further improvement of its performance by realising the potential of giving adaptive weights to the three components of the similarity measure in (7). The structural similarity measure-based tracker may not be robust to significant alteration of the tracked object.…”

Section: Discussionmentioning

confidence: 99%

“…Similarly, in the multimodal sequence bushes, Figure 6, the proposed 'structure' tracker is the most precise and the 'colour' tracker the least precise (see also Figure 2). The use of the fused video, although resulting in slightly deteriorated performance of the 'edges' tracker, can still be motivated by the fact that it retains complementary information useful both for the tracker and a human operator [21,7]: contextual information from the visible sequence and a hidden object location from the infrared sequence.…”

Section: Example Framesmentioning

confidence: 99%

See 1 more Smart Citation

Structural similarity-based object tracking in multimodality surveillance videos

Loza

Mihaylova

Bull

et al. 2007

Machine Vision and Applications

View full text Add to dashboard Cite

This paper addresses the problem of object tracking in video sequences for surveillance applications by using a recently proposed structural similarity-based image distance measure. Multimodality surveillance videos pose specific challenges to tracking algorithms, due to, for example, low or variable light conditions and the presence of spurious or camouflaged objects. These factors often cause undesired luminance and contrast variations in videos produced by infrared sensors (due to varying thermal conditions) and visible sensors (e.g. the object entering shadowy areas). Commonly used colour and edge histogram-based trackers often fail in such conditions. In contrast, the structural similarity measure reflects the distance between two video frames by jointly comparing their luminance, contrast and spatial characteristics and is sensitive to relative rather than absolute changes in the video frame. In this work, we show that the performance of a particle filter tracker is improved significantly when the structural similarity-based distance is applied instead of the conventional Bhattacharyya histogrambased distance. Extensive evaluation of the proposed algorithm is presented together with comparisons with colour, edge and mean-shift trackers using real-world surveillance video sequences from multimodal (infrared and visible) cameras.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Example Framesmentioning

confidence: 99%

Structural similarity-based object tracking in multimodality surveillance videos

Loza

Mihaylova

Bull

et al. 2007

Machine Vision and Applications

View full text Add to dashboard Cite

show abstract

“…Early fusion has been applied to define whether the audio signal is consistent with the speaker video file [9]. Pixel-level fusion has shown promising performance in video-based biometric recognition [10] as well as multiple object tracking [11]. Some authors demonstrate [12,13] demonstrated the effectiveness of the decision level fusion strategies on object tracking, video segmentation, and video event detection.…”

Section: Spatial-temporal Fusion Schemes Over Framesmentioning

confidence: 99%

Spatio-Temporal Fusion for Learning of Regions of Interests Over Multiple Video Streams

Khoshrou

Cardoso

Granger

et al. 2015

Advances in Visual Computing

View full text Add to dashboard Cite

of data captured over a network of cameras for various recognition tasks. In order to limit human labour and error, this paper presents a spatial-temporal fusion approach to accurately combine information from Region of Interest (RoI) batches captured in a multi-camera surveillance scenario. In this paper, feature-level and score-level approaches are proposed for spatial-temporal fusion of information to combine information over frames, in a framework based on ensembles of GMM-UBM (Universal Background Models). At the feature-level, features in a batch of multiple frames are combined and fed to the ensemble, whereas at the score-level the outcome of ensemble for individual frames are combined. Results indicate that feature-level fusion provides higher level of accuracy in a very efficient way. IntroductionVideo surveillance applications, such as activity recognition, are increasingly making use of multiple sensors and modalities. The fusion of multiple diverse sources of information is expected to benefit the system for the recognition of objects, persons, activities and events captured in an array of cameras.Networks of video cameras are commonly employed to monitor large areas for a variety of applications. A central issue in such networks is the tracking and recognition of individuals of interests across multiple cameras. These individuals must be recognized when leaving the Field of View (FoV) of one camera and re-identified when entering the FoV of another camera. Systems for video-to-video recognition are typically employed for person re-identification (PR). In a FoV, the appearance of an individual may be captured in reference RoIs and representative models may be learned from RoI trajectories. Then, the probe RoI may be matched against the reference model in either live (real-time monitoring) or archived (post-event analysis) [1]. In this paper, we address a PR system over wide network of cameras where no target individual enrolled to the system in advance.In such environments, where objects move and cross in the FoV of multiple cameras, it is likely to have multiple streams, recorded at different starting points with various lengths, for the same RoI of individuals (see Fig. 1a). The surveillance system must track that person across all cameras whose FoV overlap the person's path.

show abstract

“…Mihaylova et al [4] investigated how the object tracking performance was affected by the fusion quality using a particle filter, and concluded that the AVE and DT-CWT techniques had been found to outperform the other methods. Cvejic et al [5] presented an experimental approach to the assessment of the effects of pixel-level fusion on object tracking in multimodal surveillance dynamic image videos, and concluded that under some circumstances, video fusion, typically the AVE approach, was beneficial. This paper investigates how object tracking performance is affected by the dynamic image fusion quality, as compared to tracking in single modality videos.…”

Section: Introductionmentioning

confidence: 98%

A multi-cue mean-shift target tracking approach based on fuzzified region dynamic image fusion

Xiao

Wu³

2012

Sci. China Inf. Sci.

View full text Add to dashboard Cite

Traditional target tracking algorithms based on single sensor images are unstable and have low accuracy. Based on regional target detection and fuzzy region rules, a fuzzy region-based multi-sensor image fusion approach is proposed in this paper. The similarity measure weight is adapted to this dynamic image fusion algorithm, while the tracking method uses the proposed multi-cue mean-shift tracking algorithm. Three experimental results using real world image sequences are evaluated using the steady state square root mean error. The fusion and tracking experiments indicate that the proposed approach is effective and efficient when aiming at a target moving from one area to a different area, which meets the robustness and real-time requirements. CitationXiao G, Yun X, Wu J M. A multi-cue mean-shift target tracking approach based on fuzzified region dynamic image fusion.

show abstract

The Effect of Pixel-Level Fusion on Object Tracking in Multi-Sensor Surveillance Video

Cited by 56 publications

References 13 publications

Structural similarity-based object tracking in multimodality surveillance videos

Structural similarity-based object tracking in multimodality surveillance videos

Spatio-Temporal Fusion for Learning of Regions of Interests Over Multiple Video Streams

A multi-cue mean-shift target tracking approach based on fuzzified region dynamic image fusion

Contact Info

Product

Resources

About