Conditional Prior Networks for Optical Flow

Abstract

Classical computation of optical flow involves generic priors (regularizers) that capture rudimentary statistics of images, but not long-range correlations or semantics. On the other hand, fully supervised methods learn the regularity in the annotated data, without explicit regularization and with the risk of overfitting. We seek to learn richer priors on the set of possible flows that are statistically compatible with an image. Once the prior is learned in a supervised fashion, one can easily learn the full map to infer optical flow directly from two or more images, without any need for (additional) supervision. We introduce a novel architecture, called Conditional Prior Network (CPN), and show how to train it to yield a conditional prior. When used in conjunction with a simple optical flow architecture, the CPN beats all variational methods and all unsupervised learning-based ones using the same data term. It performs comparably to fully supervised ones, that however are fine-tuned to a particular dataset. Our method, on the other hand, performs well even when transferred between datasets.

Dense Depth Posterior (DDP) From Single Image and Sparse Range

“…In order to exploit a previously observed dataset D, we use a Conditional Prior Network (CPN) [35] in our framework. Conditional Prior Networks infer the probability of an optical flow given a single image.…”

“…Instead of having a separate pose network or using direct visual odometry methods, [20] uses Perspective-n-Point (PnP) [18] and Random Sample Consensus (RANSAC) [8] to obtain pose. We exploit recently introduced method to learn the conditional prior [35] to take into account scene semantics rather than using a local smoothness assumption.…”

“…To address contextual cues and scene semantics, [35] introduces a Conditional Prior Network (CPN) in the context of optical flow, which serves as a learning scheme for inferring the distribution of optical flow vectors given a single image. We leverage this technique and formulate depth completion as a maximum a-posteriori problem by factorizing it into a likelihood term and a conditional prior term, making it possible to explicitly model the semantics induced regularity of a single image.…”

Implicit and Explicit Regularization for Optical Flow Estimation

“…Liu et al [ 34 ] enriched the training data with artificial occlusions, significantly improving occlusion performance. Of great interest in this area is the work of Yang et al [ 35 ], where they proposed a Conditional Prior Network that regularizes the output based on the input. In essence, the network captures the possible motion space of a given single input image.…”

“…This is typically implemented as an additive loss term. Unlike the works in [ 17 , 18 , 19 , 20 ], where only low level image features are used, our method utilizes high-level semantic object segmentation information to calculate the regularization term and in contrast to [ 33 , 35 ] its computation is simpler and requires no training.…”