UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Loss

Meister, Simon; Hur, Junhwa; Roth, Stefan

doi:10.48550/arxiv.1711.07837

Cited by 23 publications

(49 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Together with a smoothing regularizer for the flow, this method is very effective in learning accurate predictions in nonoccluded regions, but fails when the brightness-constancy constraint is not satisfied, e.g., at occlusion boundaries across specular surfaces. Subsequent works improved these shortcomings by excluding the pixels in occluded regions from the loss using a mask obtained by forward warping [33] or a forward-backward consistency check [25]. Janai et al [17] include multiple frames for occlusion reasoning to obtain sharper flow at boundaries.…”

Section: Unsupervised Flow Estimationmentioning

confidence: 99%

“…More recently, deep learning methods have allowed to train a single neural network model to estimate optical flow for any input image pair, with a remarkable improvement in terms of accuracy and estimation efficiency. The natural evolution of the original approach by Horn and Schunk [12] has led to the unsupervised methods for optical flow [17,18,21,22,25,33]. In fact, also these methods formulate a loss function for training that consists of a range of terms, each addressing one of the key ambiguities in the optical flow estimation task.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Optical Flow Dataset Synthesis from Unpaired Images

Wälchli,

Favaro

2021

Preprint

View full text Add to dashboard Cite

The estimation of optical flow is an ambiguous task due to the lack of correspondence at occlusions, shadows, reflections, lack of texture and changes in illumination over time. Thus, unsupervised methods face major challenges as they need to tune complex cost functions with several terms designed to handle each of these sources of ambiguity. In contrast, supervised methods avoid these challenges altogether by relying on explicit ground truth optical flow obtained directly from synthetic or real data. In the case of synthetic data, the ground truth provides an exact and explicit description of what optical flow to assign to a given scene. However, the domain gap between synthetic data and real data often limits the ability of a trained network to generalize. In the case of real data, the ground truth is obtained through multiple sensors and additional data processing, which might introduce persistent errors and contaminate it. As a solution to these issues, we introduce a novel method to build a training set of pseudo-real images that can be used to train optical flow in a supervised manner. Our dataset uses two unpaired frames from real data and creates pairs of frames by simulating random warps, occlusions with super-pixels, shadows and illumination changes, and associates them to their corresponding exact optical flow. We thus obtain the benefit of directly training on real data while having access to an exact ground truth. Training with our datasets on the Sintel and KITTI benchmarks is straightforward and yields models on par or with state of the art performance compared to much more sophisticated training approaches.

show abstract

Section: Unsupervised Flow Estimationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Optical Flow Dataset Synthesis from Unpaired Images

Wälchli,

Favaro

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Although efforts have been made to seek more accurate regularization terms, OF approaches lack accuracy, especially for t-MRI motion tracking, due to the tag fading and large deformation problems [11,49]. More recently, convolutional neural networks (CNN) are trained to predict OF [16,19,20,24,26,41,31,47,53,51,48]. However, most of these works were supervised methods, with the need of a ground truth OF for training, which is nearly impossible to obtain for medical images.…”

Section: Optical Flow Approachmentioning

confidence: 99%

DeepTag: An Unsupervised Deep Learning Method for Motion Tracking on Cardiac Tagging Magnetic Resonance Images

Ye¹,

Kanski²,

Yang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Cardiac tagging magnetic resonance imaging (t-MRI) is the gold standard for regional myocardium deformation and cardiac strain estimation. However, this technique has not been widely used in clinical diagnosis, as a result of the difficulty of motion tracking encountered with t-MRI images. In this paper, we propose a novel deep learning-based fully unsupervised method for in vivo motion tracking on t-MRI images. We first estimate the motion field (INF) between any two consecutive t-MRI frames by a bi-directional generative diffeomorphic registration neural network. Using this result, we then estimate the Lagrangian motion field between the reference frame and any other frame through a differentiable composition layer. By utilizing temporal information to perform reasonable estimations on spatiotemporal motion fields, this novel method provides a useful solution for motion tracking and image registration in dynamic medical imaging. Our method has been validated on a representative clinical t-MRI dataset; the experimental results show that our method is superior to conventional motion tracking methods in terms of landmark tracking accuracy and inference efficiency. Project page is at: https://github.com/DeepTag/cardiac_ tagging_motion_estimation.

show abstract

“…The cost function is designed based on variational methods. USCNN [1], DSTFlow [27], UnFlow [24], etc. are among the methods in this category.…”

Section: Related Workmentioning

confidence: 99%

DDCNet: Deep Dilated Convolutional Neural Network for Dense Prediction

Salehi¹,

Balasubramanian²

2021

Preprint

View full text Add to dashboard Cite

Dense pixel matching problems such as optical flow and disparity estimation are among the most challenging tasks in computer vision. Recently, several deep learning methods designed for these problems have been successful. A sufficiently larger effective receptive field (ERF) and a higher resolution of spatial features within a network are essential for providing higher-resolution dense estimates. In this work, we present a systemic approach to design network architectures that can provide a larger receptive field while maintaining a higher spatial feature resolution. To achieve a larger ERF, we utilized dilated convolutional layers. By aggressively increasing dilation rates in the deeper layers, we were able to achieve a sufficiently larger ERF with a significantly fewer number of trainable parameters. We used optical flow estimation problem as the primary benchmark to illustrate our network design strategy. The benchmark results (Sintel, KITTI, and Middlebury) indicate that our compact networks can achieve comparable performance in the class of lightweight networks.

show abstract

UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Loss

Cited by 23 publications

References 0 publications

Optical Flow Dataset Synthesis from Unpaired Images

Optical Flow Dataset Synthesis from Unpaired Images

DeepTag: An Unsupervised Deep Learning Method for Motion Tracking on Cardiac Tagging Magnetic Resonance Images

DDCNet: Deep Dilated Convolutional Neural Network for Dense Prediction

Contact Info

Product

Resources

About