Robust Scale-Aware Stereo Matching Network

Okae, James; Li, Bohan; Du, Juan; Hu, Yueming

doi:10.1109/tai.2021.3115401

Cited by 7 publications

(2 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With the development of learning-based methods, the quality of 3D representation for scene perception has been steadily improved (Okae et al 2021;Xie et al 2023). Recently, stereo matching has been explored in semantic scene completion (SSC) (Li et al 2023b,a).…”

Section: Related Workmentioning

confidence: 99%

One at a Time: Progressive Multi-Step Volumetric Probability Learning for Reliable 3D Scene Perception

Li,

Sun,

Dong

et al. 2024

AAAI

View full text Add to dashboard Cite

Numerous studies have investigated the pivotal role of reliable 3D volume representation in scene perception tasks, such as multi-view stereo (MVS) and semantic scene completion (SSC). They typically construct 3D probability volumes directly with geometric correspondence, attempting to fully address the scene perception tasks in a single forward pass. However, such a single-step solution makes it hard to learn accurate and convincing volumetric probability, especially in challenging regions like unexpected occlusions and complicated light reflections. Therefore, this paper proposes to decompose the complicated 3D volume representation learning into a sequence of generative steps to facilitate fine and reliable scene perception. Considering the recent advances achieved by strong generative diffusion models, we introduce a multi-step learning framework, dubbed as VPD, dedicated to progressively refining the Volumetric Probability in a Diffusion process. Specifically, we first build a coarse probability volume from input images with the off-the-shelf scene perception baselines, which is then conditioned as the basic geometry prior before being fed into a 3D diffusion UNet, to progressively achieve accurate probability distribution modeling. To handle the corner cases in challenging areas, a Confidence-Aware Contextual Collaboration (CACC) module is developed to correct the uncertain regions for reliable volumetric learning based on multi-scale contextual contents. Moreover, an Online Filtering (OF) strategy is designed to maintain representation consistency for stable diffusion sampling. Extensive experiments are conducted on scene perception tasks including multi-view stereo (MVS) and semantic scene completion (SSC), to validate the efficacy of our method in learning reliable volumetric representations. Notably, for the SSC task, our work stands out as the first to surpass LiDAR-based methods on the SemanticKITTI dataset.

show abstract

Section: Related Workmentioning

confidence: 99%

One at a Time: Progressive Multi-Step Volumetric Probability Learning for Reliable 3D Scene Perception

Li,

Sun,

Dong

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…In addition, due to the effectiveness in modeling multi-scale contextual information, Atrous Spatial Pyramid Pooling (ASPP) or its variants have showed promising performance in semantic segmentation [26][27][28], stereo matching [29][30][31] and object detection [32][33][34]. We also add ASPP on the final layer of our SPNet to further enhance the learning capability of our network.…”

Section: Introductionmentioning

confidence: 99%

SPNet: Structure preserving network for depth completion

Luo

Fan

et al. 2023

PLoS ONE

View full text Add to dashboard Cite

Depth completion aims to predict a dense depth map from a sparse one. Benefiting from the powerful ability of convolutional neural networks, recent depth completion methods have achieved remarkable performance. However, it is still a challenging problem to well preserve accurate depth structures, such as tiny structures and object boundaries. To tackle this problem, we propose a structure preserving network (SPNet) in this paper. Firstly, an efficient multi-scale gradient extractor (MSGE) is proposed to extract useful multi-scale gradient images, which contain rich structural information that is helpful in recovering accurate depth. The MSGE is constructed based on the proposed semi-fixed depthwise separable convolution. Meanwhile, we adopt a stable gradient MAE loss (LGMAE) to provide additional depth gradient constrain for better structure reconstruction. Moreover, a multi-level feature fusion module (MFFM) is proposed to adaptively fuse the spatial details from low-level encoder and the semantic information from high-level decoder, which will incorporate more structural details into the depth modality. As demonstrated by experiments on NYUv2 and KITTI datasets, our method outperforms some state-of-the-art methods in terms of both quantitative and quantitative evaluations.

show abstract

Robust Stereo Matching Using Discriminative Multilevel Features and Multimodal Bifurcated Cost Volume Network

Okae

Qin

2023

IEEE Sensors J.

View full text Add to dashboard Cite

Robust Scale-Aware Stereo Matching Network

Cited by 7 publications

References 38 publications

One at a Time: Progressive Multi-Step Volumetric Probability Learning for Reliable 3D Scene Perception

One at a Time: Progressive Multi-Step Volumetric Probability Learning for Reliable 3D Scene Perception

SPNet: Structure preserving network for depth completion

Robust Stereo Matching Using Discriminative Multilevel Features and Multimodal Bifurcated Cost Volume Network

Contact Info

Product

Resources

About