Real-Time Multiple Object Visual Tracking for Embedded GPU Systems

Abstract

Real-time visual object tracking provides every object of interest with a unique identity and a trajectory across video frames. This is a fundamental task of many video analytics applications like traffic monitoring, or video surveillance in general. The development of real-time multiple object tracking systems on low-power edge devices as IoT nodes, without compromising accuracy, is a challenge due to the limited computing capacity of said devices. This might rule out the best in-class computer vision solutions, which nowadays are based on deep learning, and thus, they are very hardware demanding. This paper meets this challenge with a multiple object detection and tracking system that employs cutting-edge deep learning architectures on an embedded GPU while operating in real-time. For this purpose, a system has been designed that extends a joint architecture of tracking and detection by adding a module comprised of appearance-based and movement-based trackers that allow to maintain the identity of the objects of interest for longer periods of time while alleviating the burden of the detector. Our system is mapped onto an embedded GPU platform, cutting down power consumption significantly with respect to a server GPU. Tracking performance metrics show a 51.1% in Multiple Object Tracking Accuracy (MOTA) on the MOT16 dataset. This, in conjunction with a real-time processing speed of 25.2 FPS for up to 45 simultaneous objects and low power consumption of 15W, make our system an ideal solution for a wide-range of video analytics applications.Index Terms-edge computing, deep learning, multiple object tracking I. INTRODUCTIONF ROM the point of view of the type of information to be transferred to the end user, computer vision applications can be divided into two large groups: (i) those that send a continuous video stream, and (ii) those that only need to send data calculated from video analysis. The latter are especially suitable for edge computing. This is the case of video analytics scenarios, where the computer vision system makes most or all the computing on the edge, before sending the resultwhich usually does not include video, but data-to the end user [1]- [3]. Some examples of video analytics are: highway tolls, where only vehicle counting and classification in light This research was partially funded by the Spanish Ministry of Science, Innovation and Universities under grants TIN2017-84796-C2-1-R and RTI2018-097088-B-C32, and the Galician Ministry of Education, Culture and Universities under grants ED431C 2018/29, ED431C 2017/69 and accreditation 2016-2019, ED431G/08. These grants are co-funded by the European Regional Development Fund (ERDF/FEDER program).

Real-Time Siamese Multiple Object Tracker with Enhanced Proposals

“…Traditional multi-object tracking systems depend heavily on the quality of the detector employed, requiring accurate -and therefore costly-detectors to reach their full potential [10]. This poses a problem when there are realtime or hardware constraints, as it is not feasible to obtain detections for every frame.…”

Multiobject Tracking via Discriminative Embeddings for the Internet of Things

“…Multi-object tracking (MOT) is a key component in intelligent video surveillance based on IoT devices. It can provide spatial-temporal state information to assist in decision making [8]- [14], for example, the intelligent transportation system in Figure 1. However, most existing MOT methods are difficult to deploy to IoT devices with limited computing capacity and storage.…”

Tracking more than 100 arbitrary objects at 25 FPS through deep learning

“…1 Typical runtimes for accurate detectors on an NVIDIA TITAN V for an HD720 image are 23 fps for EfficientDet-D3 [5] , or 16 fps for RetinaNet [6] . Certainly, there are detectors that can run in real-time, even on embedded GPU systems, like the lightweight Tiny-YOLOv3 [7] , but they often have a poor accuracy or resort to shrinking the input image. Therefore, the most extended approach to enable real-time processing is to call the detector at a lower frame rate and perform motion estimation between detector calls, allowing the system to provide the position of the objects in all the frames.…”

“…Therefore, the most extended approach to enable real-time processing is to call the detector at a lower frame rate and perform motion estimation between detector calls, allowing the system to provide the position of the objects in all the frames. Thus, the motion estimation module feeds the affinity block when the detector is not called [7] ( Fig. 1…”