Depth Estimation Using Structured Light Flow — Analysis of Projected Pattern Flow on an Object’s Surface

Furukawa, Ryo; Sagawa, Ryusuke; Kawasaki, Hiroshi

doi:10.1109/iccv.2017.497

Cited by 26 publications

(16 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One of the first methods in the field was presented by Saxena et al [27], applying supervised learning and proposed a patchbased model and Markov Random Field (MRF). Following this work, a variety of approaches had been presented using hand crafted representations [29,18,26,11]. Recent methods use convolutional neural networks (CNN), starting from learning features for a conditional random field (CRF) model as in Liu et al [22], to learning end-to-end CNN models refined by CRFs, as in [2,40].…”

Section: Related Workmentioning

confidence: 99%

Single Image Depth Estimation Trained via Depth From Defocus Cues

Gur

Wolf

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

110

View full text Add to dashboard Cite

Estimating depth from a single RGB images is a fundamental task in computer vision, which is most directly solved using supervised deep learning. In the field of unsupervised learning of depth from a single RGB image, depth is not given explicitly. Existing work in the field receives either a stereo pair, a monocular video, or multiple views, and, using losses that are based on structure-from-motion, trains a depth estimation network. In this work, we rely, instead of different views, on depth from focus cues. Learning is based on a novel Point Spread Function convolutional layer, which applies location specific kernels that arise from the Circle-Of-Confusion in each image location. We evaluate our method on data derived from five common datasets for depth estimation and lightfield images, and present results that are on par with supervised methods on KITTI and Make3D datasets and outperform unsupervised learning approaches. Since the phenomenon of depth from defocus is not dataset specific, we hypothesize that learning based on it would overfit less to the specific content in each dataset. Our experiments show that this is indeed the case, and an estimator learned on one dataset using our method provides better results on other datasets, than the directly supervised methods.Our method relies on a novel Point Spread Function (PSF) layer, which preforms a local operation over an image, with a location dependent kernel which is computed "on-the-fly", according to the estimated parameters of the PSF at each location. More specifically, the layer receives three inputs: an all-in-focus image, estimated depth-map and camera parameters, and outputs an image at one specific focus. This image is then compared to the training images to compute a loss. Both the forward and backward operations of the layer are efficiently computed using a dedicated CUDA kernel. This layer is then used as part of a novel architecture, combining the successful ASPP architecture [5,9]. To improve the ASPP block, we add dense connections [16], followed by self-attention [42].We evaluate our method on all relevant benchmarks we were able to obtain. These include the flower lightfield dataset and the multifocus indoor and outdoor scene dataset, for which we compare the ability to generate unseen focus arXiv:2001.05036v1 [cs.CV]

show abstract

Section: Related Workmentioning

confidence: 99%

Single Image Depth Estimation Trained via Depth From Defocus Cues

Gur

Wolf

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

110

View full text Add to dashboard Cite

show abstract

“…Groundbreaking work from Saxena et al [10] introduced machine learning to estimate the depth for 2D images with monocular cues. Since then, several approaches [11], [12], [28]- [31] following this concept with different representations have been introduced.…”

Section: Related Workmentioning

confidence: 99%

Transfer2Depth: Dual Attention Network With Transfer Learning for Monocular Depth Estimation

et al. 2020

View full text Add to dashboard Cite

Monocular depth estimation poses a fundamental problem in many tasks. Although recent convolutional neural network-based methods can achieve high accuracy with very deep networks and complex architectures to exploit different cues and features, doing so not only increases the vulnerability of the model, but also increases the difficulty of convergence. Moreover, recent depth estimation methods for indoor environments are impractical for outdoor environments. In this work, we aim to develop a simple deep network structure to improve model effectiveness for depth estimation. We apply a dual attention module that can be inserted into any type of network to improve the power of representation, and additionally propose a training strategy which combines transfer learning and ordinal regression to improve training convergence. Even with a simple end-to-end encoder-decoder type of network architecture, we are able to achieve stateof-the-art performance on two of the biggest datasets for indoor and outdoor depth estimation: NYU Depth v2 and KITTI. INDEX TERMS Computer vision, deep learning, monocular depth estimation, spatial-channel attention module, transfer learning.

show abstract

“…Ultimately, Markov random field (MRF) was used to fuse the absolute features and relative features. Since then, too many approaches have been proposed to extract the monocular cues (19)(20)(21).…”

Section: Related Workmentioning

confidence: 99%

Novel Hybrid Neural Network for Dense Depth Estimation using On-Board Monocular Images

Jia

Pei

Yang

et al. 2020

Transportation Research Record: Journal of the Transportation R

View full text Add to dashboard Cite

Depth information from still 2D images plays an important role in automated driving, driving safety, and robotics. Monocular depth estimation is considered as an ill-posed and inherently ambiguous problem in general, and a tight issue is how to obtain global information efficiently since pure convolutional neural networks (CNNs) merely extract the local information. To end that, some previous works utilized conditional random fields (CRFs) to obtain the global information, but it is notoriously difficult to optimize. In this paper, a novel hybrid neural network is proposed to solve that, and concurrently a dense depth map is predicted from the monocular still image. Specifically: first, the deep residual network is utilized to obtain multi-scale local information and then feature correlation (FCL) blocks are used to correlate these features. Finally, the feature selection attention-based mechanism is adopted to fuse the multi-layer features, and the multi-layer recurrent neural networks (RNNs) are utilized with bidirectional long short-term memory (Bi-LSTM) unit as the output layer. Furthermore, a novel logarithm exponential average error (LEAE) is proposed to overcome over-weighted problem. The multi-scale feature correlation network (MFCN) is evaluated on large-scale KITTI benchmarks (LKT), which is a subset of KITTI raw dataset, and NYU depth v2. The experiments indicate that the proposed unified network outperforms existing methods. This method also updates the state-of-the-art performance on LKT datasets. Importantly, the depth estimation method can be widely used for collision risk assessment and avoidance in driving assistance systems or automated pilot systems to achieve safety in a more economical and convenient way.

show abstract

Depth Estimation Using Structured Light Flow — Analysis of Projected Pattern Flow on an Object’s Surface

Cited by 26 publications

References 16 publications

Single Image Depth Estimation Trained via Depth From Defocus Cues

Single Image Depth Estimation Trained via Depth From Defocus Cues

Transfer2Depth: Dual Attention Network With Transfer Learning for Monocular Depth Estimation

Novel Hybrid Neural Network for Dense Depth Estimation using On-Board Monocular Images

Contact Info

Product

Resources

About