Depth estimation from single monocular images using deep hybrid network

Grigorev, Aleksey; Jiang, Feng; Rho, Seungmin; Jifara, Worku; Liu, Shaohui; Sai, S. V.

doi:10.1007/s11042-016-4200-x

Cited by 17 publications

(7 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They adopted a long shortterm memory (LSTM) unit to alleviate the overfitting. Grigorev et al utilized VGG-16 as the backbone network, and two-layer RNN to obtain the global context (14,25). Chen et al had a special thought to estimate the depth of a single image by combining the RGB-D data with relative depth data (26).…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Novel Hybrid Neural Network for Dense Depth Estimation using On-Board Monocular Images

Jia

Pei

Yang

et al. 2020

Transportation Research Record: Journal of the Transportation R

View full text Add to dashboard Cite

Depth information from still 2D images plays an important role in automated driving, driving safety, and robotics. Monocular depth estimation is considered as an ill-posed and inherently ambiguous problem in general, and a tight issue is how to obtain global information efficiently since pure convolutional neural networks (CNNs) merely extract the local information. To end that, some previous works utilized conditional random fields (CRFs) to obtain the global information, but it is notoriously difficult to optimize. In this paper, a novel hybrid neural network is proposed to solve that, and concurrently a dense depth map is predicted from the monocular still image. Specifically: first, the deep residual network is utilized to obtain multi-scale local information and then feature correlation (FCL) blocks are used to correlate these features. Finally, the feature selection attention-based mechanism is adopted to fuse the multi-layer features, and the multi-layer recurrent neural networks (RNNs) are utilized with bidirectional long short-term memory (Bi-LSTM) unit as the output layer. Furthermore, a novel logarithm exponential average error (LEAE) is proposed to overcome over-weighted problem. The multi-scale feature correlation network (MFCN) is evaluated on large-scale KITTI benchmarks (LKT), which is a subset of KITTI raw dataset, and NYU depth v2. The experiments indicate that the proposed unified network outperforms existing methods. This method also updates the state-of-the-art performance on LKT datasets. Importantly, the depth estimation method can be widely used for collision risk assessment and avoidance in driving assistance systems or automated pilot systems to achieve safety in a more economical and convenient way.

show abstract

Section: Related Workmentioning

confidence: 99%

“…Unfortunately, CRFs are notoriously slow and hard to optimize. Besides, some researchers utilized the recurrent neural network (RNN) to integrate the global features and local features, but they did not design an efficient feature correlation (FCL) network (11)(12)(13)(14)(15).…”

mentioning

confidence: 99%

Novel Hybrid Neural Network for Dense Depth Estimation using On-Board Monocular Images

Jia

Pei

Yang

et al. 2020

Transportation Research Record: Journal of the Transportation R

View full text Add to dashboard Cite

show abstract

“…As far as we know, using CNN alone cannot model the longrange context well. Therefore, Grigorev et al [33] proposed a hybrid network by combining convolutional layer and ReNet layer. The ReNet layer consists of Long Short-Term Memory units (LSTMs), so the ReNet layer can obtain a global context feature representation.…”

Section: Deep Learning For Visual Odometry a Deep Learning For Depth ...mentioning

confidence: 99%

Approaches, Challenges, and Applications for Deep Visual Odometry: Toward to Complicated and Emerging Areas

Wang,

Ma,

Chen

et al. 2020

Preprint

View full text Add to dashboard Cite

Visual odometry (VO) is a prevalent way to deal with the relative localization problem, which is becoming increasingly mature and accurate, but it tends to be fragile under challenging environments. Comparing with classical geometrybased methods, deep learning-based methods can automatically learn effective and robust representations, such as depth, optical flow, feature, ego-motion, etc., from data without explicit computation. Nevertheless, there still lacks a thorough review of the recent advances of deep learning-based VO (Deep VO). Therefore, this paper aims to gain a deep insight on how deep learning can profit and optimize the VO systems. We first screen out a number of qualifications including accuracy, efficiency, scalability, dynamicity, practicability, and extensibility, and employ them as the criteria. Then, using the offered criteria as the uniform measurements, we detailedly evaluate and discuss how deep learning improves the performance of VO from the aspects of depth estimation, feature extraction and matching, pose estimation. We also summarize the complicated and emerging areas of Deep VO, such as mobile robots, medical robots, augmented reality and virtual reality, etc. Through the literature decomposition, analysis, and comparison, we finally put forward a number of open issues and raise some future research directions in this field.

show abstract

“…where M is an × diagonal matrix with th diagonal entry defined in (6), L = Λ −A represents the × Laplacian matrix at th iteration, where A denotes an × affinity matrix with entry of the th row and th column defined in (7), and Λ is an × diagonal matrix with th diagonal entry Λ defined in (8).…”

Section: Solver the Optimization Problem To Minimizementioning

confidence: 99%

“…Automatic method attempts to estimate depth from monoscopic images utilizing various cues such as defocus, texture gradients, and scattering [5]. Recently, deep-learninginspired approaches have been proposed for automatically converting 2D video/image to 3D format [5][6][7][8][9][10]. Although these methods can produce depth maps automatically, they are hard to provide robust and stable conversion results in any general content.…”

Section: Introductionmentioning

confidence: 99%

Robust Semiautomatic 2D-to-3D Conversion with Welsch M-Estimator for Data Fidelity

Yuan

et al. 2018

Mathematical Problems in Engineering

View full text Add to dashboard Cite

Semiautomatic 2D-to-3D conversion plays an important role in generating 3D contents for display. However, most existing methods assume that user scribbles are perfectly correct, and only give acceptable results when user provides accurate labels. To address this problem, Welsch M-estimator data fidelity is used to resist erroneous scribbles. The Welsch M-estimator data fidelity which is able to alleviate the influence of inaccurate scribbles has theoretical guarantee by means of its redescending property. First, the Welsch Mestimator is introduced to measure the fidelity between estimated depth and user provided depth; then local smoothness is built by using color weighted Welsch M-estimator to make neighboring pixels with similar colors have similar depth values. Finally, we solve the problem using generalized iteratively reweighted least squares algorithm. Experiments demonstrate that our method obtains competitive performance in the absence of inaccurate scribbles and outperforms the state of the art both visually and quantitatively in the presence of inaccurate scribbles.

show abstract

Depth estimation from single monocular images using deep hybrid network

Cited by 17 publications

References 26 publications

Novel Hybrid Neural Network for Dense Depth Estimation using On-Board Monocular Images

Novel Hybrid Neural Network for Dense Depth Estimation using On-Board Monocular Images

Approaches, Challenges, and Applications for Deep Visual Odometry: Toward to Complicated and Emerging Areas

Robust Semiautomatic 2D-to-3D Conversion with Welsch M-Estimator for Data Fidelity

Contact Info

Product

Resources

About