On the Interaction Between Deep Detectors and Siamese Trackers in Video Surveillance

Abstract

Visual object tracking is an important function in many real-time video surveillance applications, such as localization and spatio-temporal recognition of persons. In realworld applications, an object detector and tracker must interact on a periodic basis to discover new objects, and thereby to initiate tracks. Periodic interactions with the detector can also allow the tracker to validate and/or update its object template with new bounding boxes. However, bounding boxes provided by a state-of-the-art detector are noisy, due to changes in appearance, background and occlusion, which can cause the tracker to drift. Moreover, CNN-based detectors can provide a high level of accuracy at the expense of computational complexity, so interactions should be minimized for real-time applications. In this paper, a new approach is proposed to manage detectortracker interactions for trackers from the Siamese-FC family. By integrating a change detection mechanism into a deep Siamese-FC tracker, its template can be adapted in response to changes in a target's appearance that lead to drifts during tracking. An abrupt change detection triggers an update of tracker template using the bounding box produced by the detector, while in the case of a gradual change, the detector is used to update an evolving set of templates for robust matching. Experiments were performed using state-of-the-art Siamese-FC trackers and the YOLOv3 detector on a subset of videos from the OTB-100 dataset that mimic video surveillance scenarios. Results highlight the importance for reliable VOT of using accurate detectors. They also indicate that our adaptive Siamese trackers are robust to noisy object detections, and can significantly improve the performance of Siamese-FC tracking.

$$\text{C}^{3}\text{Net}$$: end-to-end deep learning for efficient real-time visual active camera control

“…In contrast to existing works e.g. [28,32] that follow a single target our goal is to give the same priority to all targets and attempt to follow as many as possible without a specific focus a single target. In addition, it does not require to be given an anchor target to follow and thus can be used in generic scenarios where the goal is to monitor targets in an area.…”

“…Over the last years, deep neural networks, especially Convolutional Neural Networks (CNN), have improved the state-of-the-art in static object tracking/monitoring [24,28,35,37]. Conventional solutions for active visual tracking tackle the problem by decomposing it into two or more subtasks [21], i.e., object detection typically using a machinelearning-based classifier/detector, a tracking algorithms such as Kalman filter [7], and a control output for the camera movement.…”

“…In summary, it is evident from the literature that related works make excessive use of multiple modules composed of hand-crafted models and rules that must be tuned separately and in most cases track only a single target [28]. While there has been considerable progression in utilizing deep learning for static camera tracking there has been relatively few works dealing with deep learning for active smart camera systems.…”

“…Currently existing approaches for active vision decompose the problem into separate modules, namely detection, tracking, and control and employ different algorithms for the purpose of detecting targets and then following them [17,28]. Such examples include motion detection, background modelling/subtraction, and lastly tracking by detection.…”