Image-Guided Non-Local Dense Matching With Three-Steps  Optimization

Huang, Xu; Zhang, Yongjun; Yue, Zhaoxi

doi:10.5194/isprs-annals-iii-3-67-2016

Cited by 17 publications

(14 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many other methods have been proposed for stereo depth, such as PMSC 12 , GCSVR 12 , INTS 14 , MDP 15 , ICSG 16 , which all aimed to improve the accuracy of the depth estimated from stereo vision, or to introduce a new method to estimate the depth from a stereo pair. However, there is always a trade-off between accuracy and speed for stereo vision algorithms.…”

Section: Stereo Vision Depthmentioning

confidence: 99%

“…These networks correspond to networks in Figs. [13][14][15][16] but with convolutional layers replaced with fully connected layers on the right-hand side of the network. Using different pooling sizes before the fully connected layer will cause the network to extract different levels of features, but all these configurations introduce loss of detail.…”

Section: A1: Individual Network For Depth Analysismentioning

confidence: 99%

“…Figs. [13][14][15][16][17][18][19][20] presents the networks and their corresponding compressed graphs. Two properties are assigned to each node in the graph.…”

Section: A21: Graph Contractionmentioning

confidence: 99%

“…The corresponding graphs, with assigned labels for each network, are illustrated in Figs. [13][14][15][16][17][18][19][20].…”

Section: A21: Graph Contractionmentioning

confidence: 99%

See 3 more Smart Citations

Semiparallel deep neural network hybrid architecture: first application on depth from monocular camera

2018

View full text Add to dashboard Cite

Deep neural networks are applied to a wide range of problems in recent years. In this work, Convolutional Neural Network (CNN) is applied to the problem of determining the depth from a single camera image (monocular depth). Eight different networks are designed to perform depth estimation, each of them suitable for a feature level. Networks with different pooling sizes determine different feature levels. After designing a set of networks, these models may be combined into a single network topology using graph optimization techniques. This "Semi Parallel Deep Neural Network (SPDNN)" eliminates duplicated common network layers, and can be further optimized by retraining to achieve an improved model compared to the individual topologies. In this study, four SPDNN models are trained and have been evaluated at 2 stages on the KITTI dataset. The ground truth images in the first part of the experiment are provided by the benchmark, and for the second part, the ground truth images are the depth map results from applying a state-of-the-art stereo matching method. The results of this evaluation demonstrate that using post-processing techniques to refine the target of the network increases the accuracy of depth estimation on individual mono images. The second evaluation shows that using segmentation data alongside the original data as the input can improve the depth estimation results to a point where performance is comparable with stereo depth estimation. The computational time is also discussed in this study.These authors contributed equally to this work.2 employ single camerase.g. security monitoring, automotive & consumer vision systems, and camera infrastructure for traffic and pedestrian management in smart cities. These and other smart-vision applications can greatly benefit from accurate monocular depth analysis. This challenge has been studied for a decade and is still an open research problem.Recently the idea of using neural networks to solve this problem has attracted attention. In this paper, we tackle this problem by employing a Deep Neural Network (DNN) equipped with semantic pixel-wise segmentation utilizing our recently published disparity post-processing method.This paper also introduces the use of Semi Parallel Deep Neural Networks (SPDNN). A SPDNN is a semi-parallel network topology developed using a graph theory optimization of a set of independently optimized CNNs, each targeted at a specific aspect of the more general classification problem. In 2 3 the effect of SPDNN approach on increasing convergence and improving model generalization is discussed. For the depth from monocular vision problem a fully-connected topology, optimized for fine features, is combined with a series of max-pooled topologies (2×2, 4×4 and 8×8) each optimised for coarser image features. The optimized SPDNN topology is re-trained on the full training dataset and converges to an improved set of network weights.It is worth mentioning that this network design strategy is not limited to the 'depth from monocular vision' problem, and...

show abstract

Section: Stereo Vision Depthmentioning

confidence: 99%

Section: A1: Individual Network For Depth Analysismentioning

confidence: 99%

“…Figs. [13][14][15][16][17][18][19][20] presents the networks and their corresponding compressed graphs. Two properties are assigned to each node in the graph.…”

Section: A21: Graph Contractionmentioning

confidence: 99%

“…The corresponding graphs, with assigned labels for each network, are illustrated in Figs. [13][14][15][16][17][18][19][20].…”

Section: A21: Graph Contractionmentioning

confidence: 99%

See 2 more Smart Citations

Semiparallel deep neural network hybrid architecture: first application on depth from monocular camera

2018

View full text Add to dashboard Cite

show abstract

“…Figs. 1(c) to (g) are disparity images of various 1D DIM algorithms, namely, image-guided matching (IG) (Pham and Jeon, 2013), semi-global matching (SGM) (Hirschmüller, 2008), graph cut (GC) (Kolmogorov and Zabih, 2001), image-guided non-local dense matching with three-step optimisation (INTS) (Huang et al, 2016) and stereo matching using non-texture regions and edge information (NTDE) (Kim and Kim, 2016). Several attempts (Scharstein et al, 2017;Ni et al, 2018) have been made to improve the matching results of 1D DIM in slanted or curved regions by modifying the first-order regularisation priors of SGM in accordance with the surface orientation priors.…”

Section: Introductionmentioning

confidence: 99%

Post‐filtering with surface orientation constraints for stereo dense image matching

Huang

Qin

2020

The Photogrammetric Record

Self Cite

View full text Add to dashboard Cite

Dense image matching (DIM) is a critical technique when computing accurate 3D geometric information for many photogrammetric applications. Most DIM methods adopt first‐order regularisation priors for efficient matching, which often introduce stepped biases (also called fronto‐parallel biases) into the matching results. To remove these biases and compute more accurate matching results, this paper proposes a novel post‐filtering method by adjusting the surface orientation of each pixel in the matching process. The core algorithm formulates the post‐filtering as the optimisation of a global energy function with second‐order regularisation priors. A compromise solution of the energy function is computed by breaking the optimisation into a collection of sub‐optimisations of each pixel in a local adaptive window. The proposed method was compared with several state‐of‐the‐art post‐filtering methods on indoor, aerial and satellite datasets. The comparisons demonstrate that the proposed method obtains the highest post‐filtering accuracies on all datasets.

show abstract