Efficient Coarse-to-Fine Patch Match for Large Displacement Optical Flow

Hu, Yinlin; Song, Rui; Li, Yunsong

doi:10.1109/cvpr.2016.615

Cited by 154 publications

(159 citation statements)

References 20 publications

Supporting

Mentioning

154

Contrasting

Order By: Relevance

“…Occlusion has been regarded as an important cue for estimating more accurate optical flow. Because occluded pixels do not have correspondences in the other frame, several approaches [5,12,16,23,32] aim to filter out these outliers to minimize their ill effects and apply post-processing to refine the estimates [22,33,48,67].…”

Section: Related Workmentioning

confidence: 99%

Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimation

Hur

Roth

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

292

239

View full text Add to dashboard Cite

Deep learning approaches to optical flow estimation have seen rapid progress over the recent years. One common trait of many networks is that they refine an initial flow estimate either through multiple stages or across the levels of a coarse-to-fine representation. While leading to more accurate results, the downside of this is an increased number of parameters. Taking inspiration from both classical energy minimization approaches as well as residual networks, we propose an iterative residual refinement (IRR) scheme based on weight sharing that can be combined with several backbone networks. It reduces the number of parameters, improves the accuracy, or even achieves both. Moreover, we show that integrating occlusion prediction and bi-directional flow estimation into our IRR scheme can further boost the accuracy. Our full network achieves stateof-the-art results for both optical flow and occlusion estimation across several standard datasets. Number of parameters (million) AEPE on Sintel Train Clean FlowNetC FlowNetS SpyNetOurs (PWC-Net + Occ) PWC-NetOurs (PWC-Net + Bi) Ours (IRR-PWC) LiteFlowNetOurs (PWC-Net + IRR)

show abstract

Section: Related Workmentioning

confidence: 99%

Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimation

Hur

Roth

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

292

239

View full text Add to dashboard Cite

show abstract

“…Network Architecture. The SDC network for feature description [27] was recently published and demonstrated superior performance over heuristic descriptors like SIFT [17] when applied in state-of-the-art matching algorithms (ELAS [9], SGM [12], CPM [13], FlowFields++ [25], and SceneFlowFields [26]). SDC was further shown to be more accurate and robust in patch matching compared to other feature networks.…”

Section: Sdc Featuresmentioning

confidence: 99%

An Empirical Evaluation Study on the Training of SDC Features for Dense Pixel Matching

Schuster

Wasenmüller

Unger

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

View full text Add to dashboard Cite

Training a deep neural network is a non-trivial task. Not only the tuning of hyperparameters, but also the gathering and selection of training data, the design of the loss function, and the construction of training schedules is important to get the most out of a model. In this study, we perform a set of experiments all related to these issues. The model for which different training strategies are investigated is the recently presented SDC descriptor network (stacked dilated convolution). It is used to describe images on pixel-level for dense matching tasks. Our work analyzes SDC in more detail, validates some best practices for training deep neural networks, and provides insights into training with multiple domain data.

show abstract

“…Recently, 2D optical flow benchmarks have been dominated by label-based methods [7,24], propagation methods [4,18], neural regression networks [10] and models that exploit scene-specific properties like semantics [35,3]. Most of these models do not scale well to the volumetric domain and struggle heavily with memory consumption.…”

Section: Related Workmentioning

confidence: 99%

Volumetric Flow Estimation for Incompressible Fluids Using the Stationary Stokes Equations

Lasinger

Vogel

Schindler

2017

2017 IEEE International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

In experimental fluid dynamics, the flow in a volume of fluid is observed by injecting high-contrast tracer particles and tracking them in multi-view video. Fluid dynamics researchers have developed variants of space-carving to reconstruct the 3D particle distribution at a given time-step, and then use relatively simple local matching to recover the motion over time. On the contrary, estimating the optical flow between two consecutive images is a long-standing standard problem in computer vision, but only little work exists about volumetric 3D flow. Here, we propose a variational method for 3D fluid flow estimation from multi-view data. We start from a 3D version of the standard variational flow model, and investigate different regularization schemes that ensure divergence-free flow fields, to account for the physics of incompressible fluids. Moreover, we propose a semi-dense formulation, to cope with the computational demands of large volumetric datasets. Flow is estimated and regularized at a lower spatial resolution, while the data term is evaluated at full resolution to preserve the discriminative power and geometric precision of the local particle distribution. Extensive experiments reveal that a simple sum of squared differences (SSD) is the most suitable data term for our application. For regularization, an energy whose Euler-Lagrange equations correspond to the stationary Stokes equations leads to the best results. This strictly enforces a divergence-free flow and additionally penalizes the squared gradient of the flow.

show abstract

Efficient Coarse-to-Fine Patch Match for Large Displacement Optical Flow

Cited by 154 publications

References 20 publications

Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimation

Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimation

An Empirical Evaluation Study on the Training of SDC Features for Dense Pixel Matching

Volumetric Flow Estimation for Incompressible Fluids Using the Stationary Stokes Equations

Contact Info

Product

Resources

About