Large Displacement Optical Flow from Nearest Neighbor Fields

Chen, Zhuoyuan; Jin, Hailin; Lin, Zhe; Cohen, Scott; Wu, Ying

doi:10.1109/cvpr.2013.316

Cited by 131 publications

(137 citation statements)

References 16 publications

Supporting

Mentioning

137

Contrasting

Order By: Relevance

“…The dependence on the GPU and the lack of source code are limitations. Since the publication of our conference paper, our public Matlab code has been used by both researchers to develop new optical flow algorithms (Adato et al 2011;Chen et al 2012Chen et al , 2013Jia et al 2011;Krähenbühl and Koltun 2012) and practitioners to use optical flow for different applications (Humayun et al 2011;Lin and Fisher 2012;Niu et al 2012). Currently other available opticalflow software includes (http://lmb.informatik.uni-freiburg.…”

Section: Methodsmentioning

confidence: 99%

“…At the writing of this paper (Sep. 2012), the method, Classic+NL, ranks 13th in both AAE and EPE. Several recent and high-ranking methods directly build on Classic+NL, such as layered models (Sun et al 2010b(Sun et al , 2012(Sun et al , 2013, methods with more advanced motion prior models (Chen et al 2012;Jia et al 2011), efficient optimization schemes for the non-local term (Krähenbühl and Koltun 2012), and better initialization to deal with large displacement optical flow (Chen et al 2013).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Quantitative Analysis of Current Practices in Optical Flow Estimation and the Principles Behind Them

2013

View full text Add to dashboard Cite

The accuracy of optical flow estimation algorithms has been improving steadily as evidenced by results on the Middlebury optical flow benchmark. The typical formulation, however, has changed little since the work of Horn and Schunck. We attempt to uncover what has made recent advances possible through a thorough analysis of how the objective function, the optimization method, and modern implementation practices influence accuracy. We discover that "classical" flow formulations perform surprisingly well when combined with modern optimization and implementation techniques. One key implementation detail is the median filtering of intermediate flow fields during optimization. While this improves the robustness of classical methods it actually leads to higher energy solutions, meaning that these methods are not optimizing the original objective function. To understand the principles behind this phenomenon, we derive a new objective function that formalizes the median filtering heuristic. This objective function includes a non-local smoothness term that robustly integrates flow estimates over large spatial neighborhoods. By modifying this Portions of this work were performed when DS and MJB were at Brown University. new term to include information about flow and image boundaries we develop a method that can better preserve motion details. To take advantage of the trend towards video in widescreen format, we further introduce an asymmetric pyramid downsampling scheme that enables the estimation of longer range horizontal motions. The methods are evaluated on the Middlebury, MPI Sintel, and KITTI datasets using the same parameter settings.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A Quantitative Analysis of Current Practices in Optical Flow Estimation and the Principles Behind Them

2013

View full text Add to dashboard Cite

show abstract

“…While the visual similarity between two image regions is the most important clue for finding the optical flow, it is often unreliable due to illumination changes, deformations, repetitive patterns, low texture, occlusions or blur. Hence, basically all dense optical flow methods add prior knowledge about the properties of the flow, like local smoothness assumptions [18], structure and motion adaptive assumptions [30], the assumption that motion discontinuities are more likely at image edges [26], or the assumption that the optical flow can be ap-(a) ANNF [16] (b) Our Flow Field proximated by a few motion patterns [9]. The most popular of these assumptions is the local smoothness assumption.…”

Section: Introductionmentioning

confidence: 99%

Flow Fields: Dense Correspondence Fields for Highly Accurate Large Displacement Optical Flow Estimation

Bailer

Taetz

Stricker

2015

2015 IEEE International Conference on Computer Vision (ICCV)

208

221

View full text Add to dashboard Cite

Modern large displacement optical flow algorithms usually use an initialization by either sparse descriptor matching techniques or dense approximate nearest neighbor fields. While the latter have the advantage of being dense, they have the major disadvantage of being very outlier prone as they are not designed to find the optical flow, but the visually most similar correspondence. In this paper we present a dense correspondence field approach that is much less outlier prone and thus much better suited for optical flow estimation than approximate nearest neighbor fields. Our approach is conceptually novel as it does not require explicit regularization, smoothing (like median filtering) or a new data term, but solely our novel purely data based search strategy that finds most inliers (even for small objects), while it effectively avoids finding outliers. Moreover, we present novel enhancements for outlier filtering. We show that our approach is better suited for large displacement optical flow estimation than state-of-the-art descriptor matching techniques. We do so by initializing EpicFlow (so far the best method on MPI-Sintel) with our Flow Fields instead of their originally used state-of-the-art descriptor matching technique. We significantly outperform the original EpicFlow on MPI-Sintel, KITTI and Middlebury.

show abstract

“…This technique has found recent success in many computer vision areas, such as large displacement optical flow estimation [30,31], and orderless tracking [32], etc. The ANNF estimation does not rely on the motion continuity; hence, it can provide relatively accurate motion information even though there are great changes.…”

Section: Related Workmentioning

confidence: 99%

Abrupt motion tracking via nearest neighbor field driven stochastic sampling

Zhou

et al. 2015

Neurocomputing

View full text Add to dashboard Cite

Stochastic sampling based trackers have shown good performance for abrupt motion tracking so that they have gained popularity in recent years. However, conventional methods tend to use a two-stage sampling paradigm, in which the search space needs to be uniformly explored with an inefficient preliminary sampling phase. In this paper, we propose a novel sampling-based method in the Bayesian filtering framework to address the problem. Within the framework, nearest neighbor field estimation is utilized to compute the importance proposal probabilities, which guide the Markov chain search towards promising regions and thus enhance the sampling efficiency; given the motion priors, a smoothing stochastic sampling Monte Carlo algorithm is proposed to approximate the posterior distribution through a smoothing weight-updating scheme. Moreover, to track the abrupt and the smooth motions simultaneously, we develop an abruptmotion detection scheme which can discover the presence of abrupt motions during online tracking. Extensive experiments on challenging image sequences demonstrate the effectiveness and the robustness of our algorithm in handling the abrupt motions.

show abstract

Large Displacement Optical Flow from Nearest Neighbor Fields

Cited by 131 publications

References 16 publications

A Quantitative Analysis of Current Practices in Optical Flow Estimation and the Principles Behind Them

A Quantitative Analysis of Current Practices in Optical Flow Estimation and the Principles Behind Them

Flow Fields: Dense Correspondence Fields for Highly Accurate Large Displacement Optical Flow Estimation

Abrupt motion tracking via nearest neighbor field driven stochastic sampling

Contact Info

Product

Resources

About