MeMOT: Multi-Object Tracking with Memory

Abstract

We propose an online tracking algorithm that performs the object detection and data association under a common framework, capable of linking objects after a long time span. This is realized by preserving a large spatio-temporal memory to store the identity embeddings of the tracked objects, and by adaptively referencing and aggregating useful information from the memory as needed. Our model, called MeMOT, consists of three main modules that are all Transformer-based: 1) Hypothesis Generation that produce object proposals in the current video frame; 2) Memory Encoding that extracts the core information from the memory for each tracked object; and 3) Memory Decoding that solves the object detection and data association tasks simultaneously for multi-object tracking. When evaluated on widely adopted MOT benchmark datasets, MeMOT observes very competitive performance.

ER‐DeepSORT: Pedestrian Multiobject Tracking with Enhanced Reidentification

“…The proposed JDE makes the real-time performance enhanced, but because the backbone network used the anchor-base network, it will reduce the target tracking accuracy when the anchors and features are not aligned. The recently proposed MeMOT [10] is a generalized detection and association framework using a powerful spatio-temporal memory to store the target's identity information, and can adaptively reference and aggregate useful information accordingly, this algorithm not only improved tracking accuracy but also allowed the algorithm to track targets that have disappeared for a longer period of time, but requires high device computing power.…”