Exploiting Semantic Information and Deep Matching for Optical Flow

Bai, Min; Luo, Wenjie; Kundu, Kaustav; Urtasun, Raquel

doi:10.1007/978-3-319-46466-4_10

Cited by 92 publications

(105 citation statements)

References 40 publications

Supporting

Mentioning

103

Contrasting

Unclassified

Order By: Relevance

“…In the context of image matching, deep matching networks were successfully trained for tasks such as stereo estimation [32,33], optical flow estimation [34,35], aerial image matching [36] or ground to aerial image matching [37]. In [38], a deep learning-based method is proposed to detect and match multiscale keypoints with two separated networks.…”

Section: Related Workmentioning

confidence: 99%

Exploiting Deep Matching and SAR Data for the Geo-Localization Accuracy Improvement of Optical Satellite Images

et al. 2017

Self Cite

View full text Add to dashboard Cite

Improving the geo-localization of optical satellite images is an important pre-processing step for many remote sensing tasks like monitoring by image time series or scene analysis after sudden events. These tasks require geo-referenced and precisely co-registered multi-sensor data. Images captured by the high resolution synthetic aperture radar (SAR) satellite TerraSAR-X exhibit an absolute geo-location accuracy within a few decimeters. These images represent therefore a reliable source to improve the geo-location accuracy of optical images, which is in the order of tens of meters. In this paper, a deep learning-based approach for the geo-localization accuracy improvement of optical satellite images through SAR reference data is investigated. Image registration between SAR and optical images requires few, but accurate and reliable matching points. These are derived from a Siamese neural network. The network is trained using TerraSAR-X and PRISM image pairs covering greater urban areas spread over Europe, in order to learn the two-dimensional spatial shifts between optical and SAR image patches. Results confirm that accurate and reliable matching points can be generated with higher matching accuracy and precision with respect to state-of-the-art approaches.

show abstract

Section: Related Workmentioning

confidence: 99%

Exploiting Deep Matching and SAR Data for the Geo-Localization Accuracy Improvement of Optical Satellite Images

et al. 2017

Self Cite

View full text Add to dashboard Cite

show abstract

“…The proposed algorithm successfully exploits the temporal displacement of a third image to accurately recover camera motion and also delivers high performing optical flow and disparity estimation results even though only the general motion is computed, no pre-computed optical flow is used, and no convolutional neural network (e.g. [19,5,2]) or prior 3D models (e.g. cars) are used.…”

Section: Discussionmentioning

confidence: 99%

“…The plane D p has two parameters: a 3D unit normal vectorn p = (n x p , n y p , n z p ) and disparity d p . The disparity of pixel q = (x q , y q ) using D p is given by: Dp(q) = a * xq + b * yq + c (2) where a = −n [4]. C p is a function that measures the similarity/dissimilarity of three pixels, e.g.…”

Section: Introductionmentioning

confidence: 99%

Plane labeling trinocular stereo matching with baseline recovery

Horna

Fisher

2017

2017 Fifteenth IAPR International Conference on Machine Vision Applications (MVA)

View full text Add to dashboard Cite

In this paper we present an algorithm which recovers the rigid transformation that describes the displacement of a binocular stereo rig in a scene, and uses this to include a third image to perform dense trinocular stereo matching and reduce some of the ambiguities inherent to binocular stereo. The core idea of the proposed algorithm is the assumption that the binocular baseline is projected to the third view, and thus can be used to constrain the transformation estimation of the stereo rig. Our approach shows improved performance over binocular stereo, and the accuracy of the recovered motion allows to compute optical flow from a single disparity map. These claims are validated with the KITTI 2012 data set.

show abstract

“…Recently, 2D optical flow benchmarks have been dominated by label-based methods [7,24], propagation methods [4,18], neural regression networks [10] and models that exploit scene-specific properties like semantics [35,3]. Most of these models do not scale well to the volumetric domain and struggle heavily with memory consumption.…”

Section: Related Workmentioning

confidence: 99%

Volumetric Flow Estimation for Incompressible Fluids Using the Stationary Stokes Equations

Lasinger

Vogel

Schindler

2017

2017 IEEE International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

In experimental fluid dynamics, the flow in a volume of fluid is observed by injecting high-contrast tracer particles and tracking them in multi-view video. Fluid dynamics researchers have developed variants of space-carving to reconstruct the 3D particle distribution at a given time-step, and then use relatively simple local matching to recover the motion over time. On the contrary, estimating the optical flow between two consecutive images is a long-standing standard problem in computer vision, but only little work exists about volumetric 3D flow. Here, we propose a variational method for 3D fluid flow estimation from multi-view data. We start from a 3D version of the standard variational flow model, and investigate different regularization schemes that ensure divergence-free flow fields, to account for the physics of incompressible fluids. Moreover, we propose a semi-dense formulation, to cope with the computational demands of large volumetric datasets. Flow is estimated and regularized at a lower spatial resolution, while the data term is evaluated at full resolution to preserve the discriminative power and geometric precision of the local particle distribution. Extensive experiments reveal that a simple sum of squared differences (SSD) is the most suitable data term for our application. For regularization, an energy whose Euler-Lagrange equations correspond to the stationary Stokes equations leads to the best results. This strictly enforces a divergence-free flow and additionally penalizes the squared gradient of the flow.

show abstract

Exploiting Semantic Information and Deep Matching for Optical Flow

Cited by 92 publications

References 40 publications

Exploiting Deep Matching and SAR Data for the Geo-Localization Accuracy Improvement of Optical Satellite Images

Exploiting Deep Matching and SAR Data for the Geo-Localization Accuracy Improvement of Optical Satellite Images

Plane labeling trinocular stereo matching with baseline recovery

Volumetric Flow Estimation for Incompressible Fluids Using the Stationary Stokes Equations

Contact Info

Product

Resources

About