Dense Point Trajectories by GPU-Accelerated Large Displacement Optical Flow

Abstract

Dense and accurate motion tracking is an important requirement for many video feature extraction algorithms. In this paper we provide a method for computing point trajectories based on a fast parallel implementation of a recent optical flow algorithm that tolerates fast motion. The parallel implementation of large displacement optical flow runs about 78× faster than the serial C++ version. This makes it practical to use in a variety of applications, among them point tracking. In the course of obtaining the fast implementation, we also prove that the fixed point matrix obtained in the optical flow technique is positive semi-definite. We compare the point tracking to the most commonly used motion tracker -the KLT tracker -on a number of sequences with ground truth motion. Our resulting technique tracks up to three orders of magnitude more points and is 46% more accurate than the KLT tracker. It also provides a tracking density of 48% and has an occlusion error of 3% compared to a density of 0.1% and occlusion error of 8% for the KLT tracker. Compared to the Particle Video tracker, we achieve 66% better accuracy while retaining the ability to handle large displacements while running an order of magnitude faster.

Object Segmentation by Long Term Analysis of Point Trajectories

“…This is mainly due to efficiency considerations, as the tracker in [2] could also produce denser trajectories. However, the trajectories already cover the image domain without too many larger gaps.…”

“…We argue that temporally consistent clusters over many frames can be obtained best by analyzing long term point trajectories rather than two-frame motion fields. In order to compute such trajectories, we run a tracker we developed in [2], which is based on large displacement optical flow [3]. It provides subpixel accurate tracks on one hand, and can deal with the large motion of limbs or the background on the other.…”

Online Moving Camera Background Subtraction

“…We use LDOF [18] to track dense feature points over pairs of frames. New trajectories are automatically incorporated up to a maximum number of trajectories per frame T f max .…”

Dense 3D Reconstruction with a Hand-Held Camera

“…To compute the structure of the scene we generate point correspondences using the tracker from Sundaram et al [18]. The tracker generates point trajectories based on optical flow.…”

“…We build an initial sparse reconstruction of the scene with incremental bundle adjustment and point correspondences computed with the point tracker by Sundaram et al [18]. The resulting point cloud is then integrated into an energy functional for dense reconstruction.…”