SiamRPN++: Evolution of Siamese Visual Tracking With Very Deep Networks

Abstract

Siamese network based trackers formulate tracking as convolutional feature cross-correlation between a target template and a search region. However, Siamese trackers still have an accuracy gap compared with state-of-theart algorithms and they cannot take advantage of features from deep networks, such as ResNet-50 or deeper. In this work we prove the core reason comes from the lack of strict translation invariance. By comprehensive theoretical analysis and experimental validations, we break this restriction through a simple yet effective spatial aware sampling strategy and successfully train a ResNet-driven Siamese tracker with significant performance gain. Moreover, we propose a new model architecture to perform layer-wise and depthwise aggregations, which not only further improves the accuracy but also reduces the model size. We conduct extensive ablation studies to demonstrate the effectiveness of the proposed tracker, which obtains currently the best results on five large tracking benchmarks, including OTB2015, VOT2018, UAV123, LaSOT, and TrackingNet. Our model will be released to facilitate further researches. * The first three authors contributed equally. Work done at SenseTime. Project page: https://lb1100.github.io/SiamRPN++. Recently, the Siamese network based trackers [40,1,15,42,41,24,43,52,44] have drawn much attention in the community. These Siamese trackers formulate the visual object tracking problem as learning a general similarity map by cross-correlation between the feature representations learned for the target template and the search region. To ensure tracking efficiency, the offline learned Siamese similarity function is often fixed during the running time [40,1,15]. The CFNet tracker [41] and DSiam tracker [11] update the tracking model via a running average template and a fast transformation module, respectively. The SiamRNN tracker [24] introduces the region proposal network [24] after the Siamese network and performs joint classification and regression for tracking. The DaSiamRPN tracker [52] further introduces a distractor-aware module and improves the discrimination power of the model.

Visual Tracking by Means of Deep Reinforcement Learning and an Expert Demonstrator

“…A3CTD so outperforms the expert SiamFC [2] in SS by 7.9% and MDNet [37] by 1.8%. Both our trackers are however weaker than SiamRPN++ [30]. KCF Table 3.…”

“…State-of-the-art comparison on the OTB-100 benchmark in terms of success score (SS) and precision score (PS). mark, A3CT and A3CTD have lower performance than ECO [5], MDNet [37], SiamRPN++ [30] and the expert SiamFC [2]. However, A3CT still performs better than GO-TURN [13].…”

“…Thanks to their superior representation power, in recent years various approaches based on CNNs appeared [13,10,37,5,2,30,53]. Held et al [13] and Gordon et al [10] showed how deep regression CNNs could capture the target's motion.…”

“…the previous methodologies, however they are very computationally expensive and can run at just 1 and 6 FPS respectively. Currently, the approach based on the Siamese framework is getting significant attention for their well-balanced tracking accuracy and efficiency [2,11,31,30,54,53]. These trackers formulate the visual tracking as a cross-correlation problem and are leveraging effectively from end-to-end learning of DNNs.…”

Saliency and Tracking based Semi-supervised Learning for Orbiting Satellite Segmentation

“…Martin Danelljan et al [8][9][10]19] apply multilayer features of CNN followed by a filter selection method has shown superior performance tested on tracking dataset. The Siamese network based trackers [20][21][22] formulate the visual object tracking problem as a similarity matching problem, which also achieve state-of-the-arts. However, for tracking orbiting satellite, using state-of-the-art tracking algorithms can only obtain a single tracking point or a bounding box of the object without producing a binary mask of target which can be useful to judge the status of the satellite.…”