Deep Continuous Conditional Random Fields With Asymmetric Inter-Object Constraints for Online Multi-Object Tracking

Abstract

Online Multi-Object Tracking (MOT) is a challenging problem and has many important applications including intelligence surveillance, robot navigation and autonomous driving. In existing MOT methods, individual object's movements and inter-object relations are mostly modeled separately and relations between them are still manually tuned. In addition, inter-object relations are mostly modeled in a symmetric way, which we argue is not an optimal setting. To tackle those difficulties, in this paper, we propose a Deep Continuous Conditional Random Field (DCCRF) for solving the online MOT problem in a track-bydetection framework. The DCCRF consists of unary and pairwise terms. The unary terms estimate tracked objects' displacements across time based on visual appearance information. They are modeled as deep Convolution Neural Networks, which are able to learn discriminative visual features for tracklet association. The asymmetric pairwise terms model inter-object relations in an asymmetric way, which encourages high-confidence tracklets to help correct errors of low-confidence tracklets and not to be affected by low-confidence ones much. The DCCRF is trained in an end-to-end manner for better adapting the influences of visual information as well as inter-object relations. Extensive experimental comparisons with state-of-the-arts as well as detailed component analysis of our proposed DCCRF on two public benchmarks demonstrate the effectiveness of our proposed MOT framework.

Learning Non-Uniform Hypergraph for Multi-Object Tracking

“…For a fair comparison with the state-of-the-art MOT methods, we use the reference object detections provided by the benchmark . We train the set to set recognition method (Liu, Yan, and Ouyang 2017) based on the pre-trained GoogLeNet (Szegedy et al 2015) on the training set of MOT2016 to ex- In Table 3, NT is compared with the state-of-the-art methods including EAMTT (Sanchez-Matilla, Poiesi, and Cavallaro 2016), Quad (Son et al 2017), MHT (Kim et al 2015), STAM (Chu et al 2017), NOMT (Choi 2015), AMIR (Sadeghian, Alahi, and Savarese 2017), NLPa (Levinkov et al 2017), FWT (Henschel et al 2017), LMP , INT (Lan et al 2018), and DCCRF (Zhou et al 2018). Our NT method performs on par with the state-ofthe-art trackers (e.g., FWT and LMP) in terms of tracking accuracy.…”

Deep learning in video multi-object tracking: A survey

“…In [117], Zhou et al proposed a visual displacement CNN, which learned to predict the next position of an object depending on previous positions of the objects, and the influence that an object had over other objects in the scene. That CNN was then used to predict the location of objects in the next frame, taking as input their past trajectories.…”

Robust Multi-Modality Multi-Object Tracking

“…Multi-Object Tracking Framework. Recent research of MOT primarily follows the tracking-by-detection paradigm [6,11,38,50], where object of interests is first obtained by an object detector and then linked into trajectories via data association. The data association problem could be tackled from various perspectives, e.g., min-cost flow [11,20,37], Markov decision processes (MDP) [48], partial filtering [6], Hungarian assignment [38] and graph cut [44,49].…”