Multiple Pedestrians and Vehicles Tracking in Aerial Imagery Using a Convolutional Neural Network

Abstract

In this paper, we address various challenges in multi-pedestrian and vehicle tracking in high-resolution aerial imagery by intensive evaluation of a number of traditional and Deep Learning based Single- and Multi-Object Tracking methods. We also describe our proposed Deep Learning based Multi-Object Tracking method AerialMPTNet that fuses appearance, temporal, and graphical information using a Siamese Neural Network, a Long Short-Term Memory, and a Graph Convolutional Neural Network module for more accurate and stable tracking. Moreover, we investigate the influence of the Squeeze-and-Excitation layers and Online Hard Example Mining on the performance of AerialMPTNet. To the best of our knowledge, we are the first to use these two for regression-based Multi-Object Tracking. Additionally, we studied and compared the L1 and Huber loss functions. In our experiments, we extensively evaluate AerialMPTNet on three aerial Multi-Object Tracking datasets, namely AerialMPT and KIT AIS pedestrian and vehicle datasets. Qualitative and quantitative results show that AerialMPTNet outperforms all previous methods for the pedestrian datasets and achieves competitive results for the vehicle dataset. In addition, Long Short-Term Memory and Graph Convolutional Neural Network modules enhance the tracking performance. Moreover, using Squeeze-and-Excitation and Online Hard Example Mining significantly helps for some cases while degrades the results for other cases. In addition, according to the results, L1 yields better results with respect to Huber loss for most of the scenarios. The presented results provide a deep insight into challenges and opportunities of the aerial Multi-Object Tracking domain, paving the way for future research.

Leveraging Marine Predators Algorithm with Deep Learning Object Detection for Accurate and Efficient Detection of Pedestrians

“…Region-based CNN is a class of methods to handle object recognition and localization problems that can be optimized for model performance. In [16], designed DL-related Multi-Object Tracking method AerialMPTNet compiles appearance, graphical, and temporal data by means of a Graph CNN, SNN, and LSTM for stable tracking.…”

Multiple Object Tracking in Deep Learning Approaches: A Survey

“…Azimi et al used AerialMPTNet [93] that model uses appearance, temporal, and graphical information, and which includes Siamese Neural Network, LSTM, and GNN. Siamese networks are consist of two sub-networks.…”

“…We present MOT algorithms in six categories as shown in Figure 17. Ning et al [41] Zhu et al [32] Xiang et al [93] Milan et al [19] Azimi et al [93] Zhu et al [31] Liang et al [31] Yu et al [56] Chu et al [58] Leal-Taixe et al [76] Yoon et al [73] Azimi et al [93] Wang et al [132] Lee and Kim [117] Bewley et al [33] Weng and Kitani [55] Ray and Chakraborty [44] Pegoraro and Rossi [124] Hossain and Lee [115] Huang et al [129]…”

“…Yoon et al compared the appearance from Siamese network [73]. Azimi et al applied the Siamese network in the model they used for accurate tracking AerialMPTNet [93]. Wang et al used the Siamese network in the Unsupervised learning method [132].…”