Hierarchical Discrete Distribution Decomposition for Match Density Estimation

Abstract

Explicit representations of the global match distributions of pixel-wise correspondences between pairs of images are desirable for uncertainty estimation and downstream applications. However, the computation of the match density for each pixel may be prohibitively expensive due to the large number of candidates. In this paper, we propose Hierarchical Discrete Distribution Decomposition (HD 3 ), a framework suitable for learning probabilistic pixel correspondences in both optical flow and stereo matching. We decompose the full match density into multiple scales hierarchically, and estimate the local matching distributions at each scale conditioned on the matching and warping at coarser scales. The local distributions can then be composed together to form the global match density. Despite its simplicity, our probabilistic method achieves state-ofthe-art results for both optical flow and stereo matching on established benchmarks. We also find the estimated uncertainty is a good indication of the reliability of the predicted correspondences.

Mono-SF: Multi-View Geometry Meets Single-View Depth for Monocular Scene Flow Estimation of Dynamic Traffic Scenes

“…The results of the EveryPixel approach are stated in their paper [70] (D2 metric is excluded as it seems to be inconsistent). As a second category, single-view depth estimation ('LRC [19]' or 'DORN [14]') and optical flow estimation ('MirrorFlow [30] and 'HD 3 -F [72]' used parts of the dataset for training, these methods are disregarded for ranking. The third group comprises the multi-body or non-rigid SfM-based methods DMDE [51] and S.Soup [36].…”

Unsupervised Domain Adaptation for Depth Prediction from Images

“…To tackle this issue, a large synthetic dataset [2] was created and deployed for training, with KITTI images used to address the domain shift issue arising when running the network on real imagery. Although DispNetC did not reach the top rank on KITTI, it inspired other end-to-end models [5], [17], [18] which, in turn, were able to achieve state-ofthe-art performance. Along a similar research line, some authors deploy 3D convolutions to exploit geometry and context [6], [19], [20], [21].…”

“…Purposely, the training procedure for end-to-end stereo architectures relies on an initial optimization based on a large synthetic dataset [2] followed by fine-tuning on, possibly many, image pairs with groundtruth sourced from the target domain. As a matter of fact, the popular KITTI benchmarks [3], [4] witness the supremacy of deep stereo architectures [5], [6], while this is quite less evident in the Middlebury benchmark [7], where traditional, hand-crafted algorithms [8], [9] still keep the top rankings on the leaderboards due to the smaller amount of images available for training. Deep learning did also dramatically boost development and performance of depth-from-mono architectures, which can predict depth from just one image and, thus, be potentially deployed on the far broader range of devices equipped with a single camera.…”

PLIN: A Network for Pseudo-LiDAR Point Cloud Interpolation

“…Instead of directly investigating the optical flow on sparse depth maps, we learn the motion relationship on dense color images due to their abundant and precise contextual information. Recently, deep neural networks have shown excellent performance in optical flow estimation [6], [7], [25]. Given two consecutive color images C t−1 and C t+1 , video interpolation [1], [8], [14] aims to generate the intermediate color image C t using a bidirectional optical flow:…”