Real-Time Semantic Segmentation-Based Stereo Reconstruction

Miclea, Vlad-Cristian; Nedevschi, Sergiu

doi:10.1109/tits.2019.2913883

Cited by 33 publications

(9 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The task aims to find corresponding pixels in a stereo pair, and the distance between corresponding pixels is known as disparity (Hartley and Zisserman 2003). Based on the epipolar geometry, stereo matching enables stable depth perception from estimated disparity, hence it has been applied to further applications such as scene understanding (Franke and Joos 2000;Miclea and Nedevschi 2019;Zhang et al 2010), object detection Li et al 2018Li et al , 2019Fang et al 2018), visual odometry (Wang et al 2017;Zhu 2017), and SLAM (Engel et al 2015;Gomez-Ojeda et al 2019).…”

Section: Introductionmentioning

confidence: 99%

AdaStereo: An Efficient Domain-Adaptive Stereo Matching Approach

et al. 2022

View full text Add to dashboard Cite

Recently, records on stereo matching benchmarks are constantly broken by end-to-end disparity networks. However, the domain adaptation ability of these deep models is quite limited. Addressing such problem, we present a novel domain-adaptive approach called AdaStereo that aims to align multi-level representations for deep stereo matching networks. Compared to previous methods, our AdaStereo realizes a more standard, complete and effective domain adaptation pipeline. Firstly, we propose a non-adversarial progressive color transfer algorithm for input image-level alignment. Secondly, we design an efficient parameter-free cost normalization layer for internal feature-level alignment. Lastly, a highly related auxiliary task, self-supervised occlusion-aware reconstruction is presented to narrow the gaps in output space. We perform intensive ablation studies and break-down comparisons to validate the effectiveness of each proposed module. With no extra inference overhead and only a slight increase in training complexity, our AdaStereo models achieve state-of-the-art cross-domain performance on multiple benchmarks, including KITTI, Middlebury, ETH3D and DrivingStereo, even outperforming some state-of-the-art disparity networks finetuned with target-domain ground-truths. Moreover, based on two additional evaluation metrics, the superiority of our domain-adaptive stereo matching pipeline is further uncovered from more perspectives. Finally, we demonstrate that our method is robust to various domain adaptation settings, and can be easily integrated into quick adaptation application scenarios and real-world deployments.

show abstract

Section: Introductionmentioning

confidence: 99%

AdaStereo: An Efficient Domain-Adaptive Stereo Matching Approach

et al. 2022

View full text Add to dashboard Cite

show abstract

“…In this section, we use ResNet50 and Xception39 as the backbone to analyze the enhancement of the overall network by different modules in BSDNet, as shown in Table. 4. BS indicates that only the backbone is used for the experiment, "BS+BSDModule" indicates that a BSDModule is added to the backbone, where the expansion factor e of the BSDModule is 5.…”

Section: ) Accuracy Analysismentioning

confidence: 99%

BSDNet: Balanced Sample Distribution Network for Real-Time Semantic Segmentation of Road Scenes

Zeng

Yue

et al. 2021

IEEE Access

View full text Add to dashboard Cite

In recent years, semantic segmentation based on deep convolutional neural networks has developed rapidly. However, it is still a challenge to balance the computing cost and segmentation performance for the current semantic segmentation methods. This paper proposes a lightweight real-time semantic segmentation model Balanced Sample Distribution Network (BSDNet), to solve this problem. In BSDNet, we introduce the Balanced Sample Distribution Module (BSDModule) to balance the sampling distribution of convolution and obtain features with a larger receptive field. To optimize the segmentation effect, we introduce a Shuffle Channel Attention Module (SCAModule) to enhance the interaction of channel features at the cost of a small number of parameters. BSDModule and SCAModule are lightweight and flexible and can adapt to different types of network structures. Extensive experiments on CityScapes and CamVid show that the proposed method can balance the computing cost and segmentation performance. Specifically, on the CityScapes with 512×1024 resolution, BSDNet-Xception39 achieves 68.3% MIoU and 84.6 FPS with only 1.2M parameters. INDEX TERMSReal-time semantic segmentation, balanced sample distribution module, shuffle channel attention module.

show abstract

“…Semantically-Guided Depth Estimation Complementing self-supervised depth estimation by the additional prediction of semantic or instance segmentation in cross-task guidance approaches has been shown to increase the performance of both prediction modalities [7], [17], [8], [26], [27], [28]. Either the segmentation masks are given as an additional input to the network [28], or they are used to predict relative poses for the single dynamic objects to counterpoise their movement between consecutive frames [7], [17].…”

Section: Related Workmentioning

confidence: 99%

SVDistNet: Self-Supervised Near-Field Distance Estimation on Surround View Fisheye Cameras

Kumar¹,

Klingner²,

Yogamani³

et al. 2021

Preprint

View full text Add to dashboard Cite

A 360 • perception of scene geometry is essential for automated driving, notably for parking and urban driving scenarios. Typically, it is achieved using surround-view fisheye cameras, focusing on the near-field area around the vehicle. The majority of current depth estimation approaches focus on employing just a single camera, which cannot be straightforwardly generalized to multiple cameras. The depth estimation model must be tested on a variety of cameras equipped to millions of cars with varying camera geometries. Even within a single car, intrinsics vary due to manufacturing tolerances. Deep learning models are sensitive to these changes, and it is practically infeasible to train and test on each camera variant. As a result, we present novel camera-geometry adaptive multiscale convolutions which utilize the camera parameters as a conditional input, enabling the model to generalize to previously unseen fisheye cameras. Additionally, we improve the distance estimation by pairwise and patchwise vector-based self-attention encoder networks. We evaluate our approach on the Fisheye WoodScape surround-view dataset, significantly improving over previous approaches. We also show a generalization of our approach across different camera viewing angles and perform extensive experiments to support our contributions. To enable comparison with other approaches, we evaluate the front camera data on the KITTI dataset (pinhole camera images) and achieve state-of-the-art performance among self-supervised monocular methods. An overview video with qualitative results is provided at https://youtu.be/bmX0UcU9wtA. Baseline code and dataset will be made public 1 .

show abstract

Real-Time Semantic Segmentation-Based Stereo Reconstruction

Cited by 33 publications

References 49 publications

AdaStereo: An Efficient Domain-Adaptive Stereo Matching Approach

AdaStereo: An Efficient Domain-Adaptive Stereo Matching Approach

BSDNet: Balanced Sample Distribution Network for Real-Time Semantic Segmentation of Road Scenes

SVDistNet: Self-Supervised Near-Field Distance Estimation on Surround View Fisheye Cameras

Contact Info

Product

Resources

About