MVLidarNet: Real-Time Multi-Class Scene Understanding for Autonomous Driving Using Multiple Views

Chen, Ke; Oldja, Ryan; Smolyanskiy, Nikolai; Birchfield, Stan; Popov, Alexander; Wehr, David; Eden, Ibrahim; Pehserl, Joachim

doi:10.48550/arxiv.2006.05518

Cited by 5 publications

(4 citation statements)

References 22 publications

(29 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Among these three focused methods, [27] scores best in terms of mIoU and mAP values, with superior FPS, which are 31.30, 65.50, and 65.5, respectively. The reported results over the SemanticKITTI dataset indicate a better performance of the method in [51], achieving well-balanced mIoU and FPS scores, i.e., 52.20 and 92, respectively. Finally, the method in [65] performs comparatively better than the one in [107], by offering best values of the mIoU, mAP, and FPS scores (75.70, 83.60, and 19.5, respectively).…”

Section: Quantitative Analysis Of Scene Segmentation Methods For Admentioning

confidence: 84%

Vision-Based Semantic Segmentation in Scene Understanding for Autonomous Driving: Recent Achievements, Challenges, and Outlooks

Muhammad

Hussain

Ullah

et al. 2022

IEEE Trans. Intell. Transport. Syst.

View full text Add to dashboard Cite

Scene understanding plays a crucial role in autonomous driving by utilizing sensory data for contextual information extraction and decision making. Beyond modeling advances, the enabler for vehicles to become aware of their surroundings is the availability of visual sensory data, which expand the vehicular perception and realizes vehicular contextual awareness in real-world environments. Research directions for scene understanding pursued by related studies include person/vehicle detection and segmentation, their transition analysis, lane change, and turns detection, among many others.

show abstract

Section: Quantitative Analysis Of Scene Segmentation Methods For Admentioning

confidence: 84%

Vision-Based Semantic Segmentation in Scene Understanding for Autonomous Driving: Recent Achievements, Challenges, and Outlooks

Muhammad

Hussain

Ullah

et al. 2022

IEEE Trans. Intell. Transport. Syst.

View full text Add to dashboard Cite

show abstract

“…Multi-view fusion-based methods combine voxel-based, projection-based and/or point-wise operations for LiDAR point clouds segmentation. To extract more semantic information, some recent methods [35], [36], [37], [38], [39], [40], [41], [7], [8] blend two or more different views together. For instance, [38], [39] combine point-wise information from BEV and range-image in early-stage, and then feed it to the subsequent network.…”

Section: B Lidar Point Cloud Semantic Segmentationmentioning

confidence: 99%

LIF-Seg: LiDAR and Camera Image Fusion for 3D LiDAR Semantic Segmentation

Zhao,

Zhou,

Zhu

et al. 2021

Preprint

View full text Add to dashboard Cite

Camera and 3D LiDAR sensors have become indispensable devices in modern autonomous driving vehicles, where the camera provides the fine-grained texture, color information in 2D space and LiDAR captures more precise and farther-away distance measurements of the surrounding environments. The complementary information from these two sensors makes the two-modality fusion be a desired option. However, two major issues of the fusion between camera and LiDAR hinder its performance, i.e., how to effectively fuse these two modalities and how to precisely align them (suffering from the weak spatiotemporal synchronization problem). In this paper, we propose a coarse-to-fine LiDAR and camera fusion-based network (termed as LIF-Seg) for LiDAR segmentation. For the first issue, unlike these previous works fusing the point cloud and image information in a one-to-one manner, the proposed method fully utilizes the contextual information of images and introduces a simple but effective early-fusion strategy. Second, due to the weak spatiotemporal synchronization problem, an offset rectification approach is designed to align these two-modality features. The cooperation of these two components leads to the success of the effective camera-LiDAR fusion. Experimental results on the nuScenes dataset show the superiority of the proposed LIF-Seg over existing methods with a large margin. Ablation studies and analyses demonstrate that our proposed LIF-Seg can effectively tackle the weak spatiotemporal synchronization problem.

show abstract

“…In comparison, 2D BEV or RV grid based methods are efficient but only use a single view (either BEV or RV) for processing Li-DAR data. Recent work has investigated the use of multiple views [20,21,22,23,24] and shown that the complementary benefits of both views improve performance. However, these methods use only one frame of LiDAR data and only solve perception tasks such as object detection and semantic segmentation.…”

Section: Lidar Representationmentioning

confidence: 99%

MVFuseNet: Improving End-to-End Object Detection and Motion Forecasting through Multi-View Fusion of LiDAR Data

Laddha¹,

Gautam²,

Palombo³

et al. 2021

Preprint

View full text Add to dashboard Cite

In this work, we propose MVFuseNet, a novel end-toend method for joint object detection and motion forecasting from a temporal sequence of LiDAR data. Most existing methods operate in a single view by projecting data in either range view (RV) or bird's eye view (BEV). In contrast, we propose a method that effectively utilizes both RV and BEV for spatio-temporal feature learning as part of a temporal fusion network as well as for multi-scale feature learning in the backbone network. Further, we propose a novel sequential fusion approach that effectively utilizes multiple views in the temporal fusion network. We show the benefits of our multi-view approach for the tasks of detection and motion forecasting on two large-scale self-driving data sets, achieving state-of-the-art results. Furthermore, we show that MVFusenet scales well to large operating ranges while maintaining real-time performance.

show abstract

MVLidarNet: Real-Time Multi-Class Scene Understanding for Autonomous Driving Using Multiple Views

Cited by 5 publications

References 22 publications

Vision-Based Semantic Segmentation in Scene Understanding for Autonomous Driving: Recent Achievements, Challenges, and Outlooks

Vision-Based Semantic Segmentation in Scene Understanding for Autonomous Driving: Recent Achievements, Challenges, and Outlooks

LIF-Seg: LiDAR and Camera Image Fusion for 3D LiDAR Semantic Segmentation

MVFuseNet: Improving End-to-End Object Detection and Motion Forecasting through Multi-View Fusion of LiDAR Data

Contact Info

Product

Resources

About