Online Multiple Object Tracking Based on Open-Set Few-Shot Learning

Abstract

How to make an online tracking model effectively adapt to newly appearing objects and object disappearance as well as appearance variations of target objects from few examples is an essential issue in multiple object tracking (MOT). Learning target appearances from few examples is a few-shot classification problem, while identifications of newly appearing objects and object disappearance has the aspect of open-set classification. In this work, we regard online MOT as open-set few-show classification to address both learning from few examples (few-shot classification) and unknown classes such as new objects (open-set classification). Specifically, we develop an embedding neural network, called VOFNet, consisting of convolutional and recurrent parts, to perform open-set few-shot classification. The convolutional part constructs a feature from an example of a target object and the recurrent part determines a representative feature of a target object from few examples. Then VOFNet is trained to provide effective features for openset few-shot classification. Finally, we develop an online multiple object tracker based on the combination of VOFNet and the bipartite matching. The proposed tracker achieves 49.2 multiple object tracking accuracy (MOTA) with 28.9 frames per second on MOT17 dataset, which shows a significantly better trade-off between the accuracy and the speed than the existing algorithms. For example, the proposed algorithm yields about 3.17 times faster speed with 0.99 times lower accuracy than recent existing MOT algorithm [1]. INDEX TERMS Multiple object tracking, online tracking, open-set classification, few-shot classification.

DBSCAN-Based Tracklet Association Annealer for Advanced Multi-Object Tracking

“…The MOT technique based on the features of the detected object was proposed after the development of the DNN, and it was followed by feature vector embedding [ 20 , 21 ]. Although various similarity models have been proposed, such as modifying the convolutional neural network (CNN) structure [ 22 , 23 ] or vector embedding through fusion with long short-term memory [ 24 , 25 ], many studies still use feature vectors and graph models extracted through reidentification-based CNN [ 26 , 27 ]. They have higher robustness than graph models in an environment where temporal occlusion or noise exists, but vast amounts of data must be secured for learning.…”