FollowMe: Efficient Online Min-Cost Flow Tracking with Bounded Memory and Computation

Abstract

One of the most popular approaches to multi-target tracking is tracking-by-detection. Current min-cost flow algorithms which solve the data association problem optimally have three main drawbacks: they are computationally expensive, they assume that the whole video is given as a batch, and they scale badly in memory and computation with the length of the video sequence. In this paper, we address each of these issues, resulting in a computationally and memory-bounded solution. First, we introduce a dynamic version of the successive shortest-path algorithm which solves the data association problem optimally while reusing computation, resulting in significantly faster inference than standard solvers. Second, we address the optimal solution to the data association problem when dealing with an incoming stream of data (i.e., online setting). Finally, we present our main contribution which is an approximate online solution with bounded memory and computation which is capable of handling videos of arbitrarily length while performing tracking in real time. We demonstrate the effectiveness of our algorithms on the KITTI and PETS2009 benchmarks and show state-of-the-art performance, while being significantly faster than existing solvers.

Machine vision methods for analyzing social interactions

“…However, as optimization cost grows super-linearly in the number of unknowns, solving such a large problem can be too computationally expensive. Many approaches have been developed for improving the efficiency of these algorithms, including making greedy decisions to reduce the problem size and developing more efficient and/or approximate solvers (Lenz et al, 2015).…”

Robust Multi-Modality Multi-Object Tracking

“…Recent research of MOT primarily follows the tracking-by-detection paradigm [6,11,38,50], where object of interests is first obtained by an object detector and then linked into trajectories via data association. The data association problem could be tackled from various perspectives, e.g., min-cost flow [11,20,37], Markov decision processes (MDP) [48], partial filtering [6], Hungarian assignment [38] and graph cut [44,49]. However, most of these methods are not trained in an end-to-end manner thus many parameters are heuristic (e.g., weights of costs) and susceptible to local optima.…”

AlignNet-3D: Fast Point Cloud Registration of Partially Observed Objects

“…Even though high-level path planning requires motion perception in 3D space, the task of continuous 3D state estimation of tracked targets is often neglected in existing approaches. Methods based on network flow [52,39,22,31] only focus on discrete optimization and do not estimate the continuous state of targets. Several approaches cast state estimation as inference in linear dynamical systems such as Kalman filters.…”

“…Several approaches cast state estimation as inference in linear dynamical systems such as Kalman filters. Most often, the target state is parametrized as a 2D bounding box in the image domain (vision-based methods [20,22,23,38]) or approximated by the center of mass or 3D bounding box in case of stereo-based [21,29,24,9] or LiDAR-based methods [42,30,8,18,26]. While such coarse approximations are perfectly reasonable in case the target is not in our direct proximity, it is imperative that we estimate motion precisely in the vehicle close-range -imprecisions in this range can lead to catastrophic consequences.…”