An Improved End-to-End Multi-Target Tracking Method Based on Transformer Self-Attention

Abstract

Current multi-target multi-camera tracking algorithms demand increased requirements for re-identification accuracy and tracking reliability. This study proposed an improved end-to-end multi-target tracking algorithm that adapts to multi-view multi-scale scenes based on the self-attentive mechanism of the transformer’s encoder–decoder structure. A multi-dimensional feature extraction backbone network was combined with a self-built raster semantic map which was stored in the encoder for correlation and generated target position encoding and multi-dimensional feature vectors. The decoder incorporated four methods: spatial clustering and semantic filtering of multi-view targets; dynamic matching of multi-dimensional features; space–time logic-based multi-target tracking, and space–time convergence network (STCN)-based parameter passing. Through the fusion of multiple decoding methods, multi-camera targets were tracked in three dimensions: temporal logic, spatial logic, and feature matching. For the MOT17 dataset, this study’s method significantly outperformed the current state-of-the-art method by 2.2% on the multiple object tracking accuracy (MOTA) metric. Furthermore, this study proposed a retrospective mechanism for the first time and adopted a reverse-order processing method to optimize the historical mislabeled targets for improving the identification F1-score (IDF1). For the self-built dataset OVIT-MOT01, the IDF1 improved from 0.948 to 0.967, and the multi-camera tracking accuracy (MCTA) improved from 0.878 to 0.909, which significantly improved the continuous tracking accuracy and reliability.

Global-Local and Occlusion Awareness Network for Object Tracking in UAVs

“…CNNbased multi-object tracking algorithm have limitations due to their reliance on local perception, difficulty in modeling long-term dependencies and capturing global features, and the negative impact of spatial invariance and pooling operations on tracking accuracy. Transformer has shown great success in various vision tasks due to its ability to capture global features and long-term modeling [32]- [36]. Therefore, using only CNN or Transformer as the detector of the tracking network cannot adequately capture global and local features [51], [52], especially in the UAV scenario.…”