Robust Visual Tracking by Integrating Multiple Cues Based on Co-Inference Learning

Abstract

Abstract. Visual tracking can be treated as a parameter estimation problem that infers target states based on image observations from video sequences. A richer target representation would incur better chances of successful tracking in cluttered and dynamic environments, and thus enhance the robustness. Richer representations can be constructed by either specifying a detailed model of a single cue or combining a set of rough models of multiple cues. Both approaches increase the dimensionality of the state space, which results in a dramatic increase of computation. To investigate the integration of rough models from multiple cues and to explore computationally efficient algorithms, this paper formulates the problem of multiple cue integration and tracking in a probabilistic framework based on a factorized graphical model. Structured variational analysis of such a graphical model factorizes different modalities and suggests a co-inference process among these modalities. Based on the importance sampling technique, a sequential Monte Carlo algorithm is proposed to provide an efficient simulation and approximation of the co-inferencing of multiple cues. This algorithm runs in real-time at around 30Hz. Our extensive experiments show that the proposed algorithm performs robustly in a large variety of tracking scenarios. The approach presented in this paper has the potential to solve other problems including sensor fusion problems.

Disagreement-Based Multi-system Tracking

“…A number of tracking methods have been proposed to perform fusion [13,14,18,19,16,15,17]. Different from [13,17] where multiple parts were tracked and correlated, we deal with a single target.…”

“…Different from [13,17] where multiple parts were tracked and correlated, we deal with a single target. In [14,16] multiple trackers were fused but these trackers represent different features and they were directly combined.…”

“…It involves multiple classifiers which label the unlabeled data to update and improve each other [12]. From a different angle, the use of multiple classifiers can be viewed as a fusion problem and it has been shown that fusing complementary features in a tracking system often leads to enhanced performances [13][14][15]. However, less efforts have been made in learning to fuse well-developed existing algorithms through semi-supervised learning; we will see later (in both theory and experiments) that a disagreement-based fusion significantly improves the performance over direct combination of features/systems [14,16,17].…”

Visual Tracking by Affine Kernel Fitting Using Color and Object Boundary

“…In [18] two trackers, a region tracker and an edge tracker, ran in parallel and performed mutual corrections based on their confidence measures. Another example are the co-inference algorithms, developed in [19] to combine trackers of different modalities.…”

A Probabilistic Approach to Integrating Multiple Cues in Visual Tracking

“…Many methods assume that the cues are conditionally independent [7][8][9][10], while more sophisticated methods model existing dependencies explicitly by e.g. graphical models [11]. This paper presents a novel probabilistic approach to integrating multiple cues in visual tracking.…”