Trajectory Factory: Tracklet Cleaving and Re-Connection by Deep Siamese Bi-GRU for Multiple Object Tracking

Abstract

Multi-Object Tracking (MOT) is a challenging task in the complex scene such as surveillance and autonomous driving. In this paper, we propose a novel tracklet processing method to cleave and re-connect tracklets on crowd or longterm occlusion by Siamese Bi-Gated Recurrent Unit (GRU). The tracklet generation utilizes object features extracted by CNN and RNN to create the high-confidence tracklet candidates in sparse scenario. Due to mis-tracking in the generation process, the tracklets from different objects are split into several sub-tracklets by a bidirectional GRU. After that, a Siamese GRU based tracklet re-connection method is applied to link the sub-tracklets which belong to the same object to form a whole trajectory. In addition, we extract the tracklet images from existing MOT datasets and propose a novel dataset to train our networks. The proposed dataset contains more than 95160 pedestrian images. It has 793 different persons in it. On average, there are 120 images for each person with positions and sizes. Experimental results demonstrate the advantages of our model over the state-of-the-art methods on MOT16.

Deep learning in video multi-object tracking: A survey

“…Both will be further discussed in section 3.3. Ma et al [116] also trained a Siamese CNN in order to extract visual features from tracked pedestrians in their model, which is explained in detail in section 3.4.1.…”

“…Ma et al [116] used a bidirectional GRU RNN to decide where to split tracklets. The algorithm proceeded in three main stages: a tracklet generation step, that included a NMS step to remove redundant detections and then employed the Hungarian algorithm with appearance and motion affinity together to form high-confidence tracklets; then, a tracklet cleaving step was performed: since a tracklet might contain an ID switch error due to occlusions, this step aimed to split the tracklets at the point where the ID switch happened, in order to obtain two separate tracklets that contained the same identity; finally, a tracklet reconnection step was employed, using a customized association algorithm that made use of features extracted by a Siamese bidirectional GRU.…”

OneShotDA: Online Multi-Object Tracker With One-Shot-Learning-Based Data Association

“…The classification results for the validation set indicate that OneShotDA is trained properly and makes precise association predictions. Finally, we present performance comparisons between the OneShotDA tracker and existing state-of-the-art methods such as HCC [17], LMP [19], GCRA [36], KCF16 [37], MOTDT [30], JBNOT [20], eHAF17 [39], TLMHT [22], EAGS16 [38], MHT_DAM [13], MHT_bLSTM [23], and EDMT17 [40]. These methods were evaluated on the MOTChallenge server.…”

Eliminating Exposure Bias and Metric Mismatch in Multiple Object Tracking

“…We remove this bias by introducing a much more exhaustive, yet computationally feasible, approach to exploiting the data while training the model. To this end, during training, we do not limit ourselves to only using tracklets made of detections of one or two people as in [40,35,48]. Instead, we consider any grouping of tracklets produced by the tracking algorithm to be a potential trajectory but prevent a combinatorial explosion by controlling the number of tracklets that share many common detections.…”