PillarSegNet: Pillar-based Semantic Grid Map Estimation using Sparse LiDAR Data

Fei, Juncong; Peng, Kunyu; Heidenreich, Philipp; Bieder, Frank; Stiller, Christoph

doi:10.1109/iv48863.2021.9575694

Cited by 29 publications

(13 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Both approaches [22], [24] use a multi-layer grid with handcrafted features as input data and the same semantic grid representation as output. Relying on the same label data, Fei et al [26] propose to generate pillar features [12] and fuse them with an observability map as input for an encoder-decoder network. Their approach receive improved results compared to [24], especially for small objects.…”

Section: Related Workmentioning

confidence: 99%

“…Their approach receive improved results compared to [24], especially for small objects. The difference of the dense semantic grid maps predicted in [22], [24], [26] compared to a dynamic occupancy grid map is the missing information whether obstacles are static or moving as well as the modeling of occlusions caused by dynamic objects. In this work, we combine the prediction of a dynamic occupancy grid map with the semantic classification of occupied cells, based on lidar input data.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

A Multi-Task Recurrent Neural Network for End-to-End Dynamic Occupancy Grid Mapping

Schreiber¹,

Belagiannis²,

Gläser³

et al. 2022

Preprint

View full text Add to dashboard Cite

A common approach for modeling the environment of an autonomous vehicle are dynamic occupancy grid maps, in which the surrounding is divided into cells, each containing the occupancy and velocity state of its location. Despite the advantage of modeling arbitrary shaped objects, the used algorithms rely on hand-designed inverse sensor models and semantic information is missing. Therefore, we introduce a multi-task recurrent neural network to predict grid maps providing occupancies, velocity estimates, semantic information and the driveable area. During training, our network architecture, which is a combination of convolutional and recurrent layers, processes sequences of raw lidar data, that is represented as bird's eye view images with several height channels. The multi-task network is trained in an end-to-end fashion to predict occupancy grid maps without the usual preprocessing steps consisting of removing ground points and applying an inverse sensor model. In our evaluations, we show that our learned inverse sensor model is able to overcome some limitations of a geometric inverse sensor model in terms of representing object shapes and modeling freespace. Moreover, we report a better runtime performance and more accurate semantic predictions for our end-to-end approach, compared to our network relying on measurement grid maps as input data.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

A Multi-Task Recurrent Neural Network for End-to-End Dynamic Occupancy Grid Mapping

Schreiber¹,

Belagiannis²,

Gläser³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Observing that there is a sufficient overlap between measurements, it should be reasonable to expect that fusing successive estimates of the neural network, as the vehicle moves forward, should improve overall semantic segmentation. Although researchers have investigated this idea, they often limit applications to indoor environments [39]- [41] or 2D representations of outdoor environments [11], [42], [43].…”

Section: B Information Fusionmentioning

confidence: 99%

“…A map of the environment can be used to establish situational awareness, localize the autonomous vehicle (AV), plan safe trajectories taking into account the geometry of the road, traffic rules, and the position of surrounding objects [1], [2]. Endowing the map with semantic information can further enhance understanding of the surrounding environment [3]. If the map is to contribute to the tasks above, the system has to have the ability to analyze the map in real-time.…”

Section: Introductionmentioning

confidence: 99%

Semantic 3D Grid Maps for Autonomous Driving

Khoche¹,

Wozniak²,

Duberg³

2022

Preprint

View full text Add to dashboard Cite

Maps play a key role in rapidly developing area of autonomous driving. We survey the literature for different map representations and find that while the world is threedimensional, it is common to rely on 2D map representations in order to meet real-time constraints. We believe that high levels of situation awareness require a 3D representation as well as the inclusion of semantic information. We demonstrate that our recently presented hierarchical 3D grid mapping framework UFOMap meets the real-time constraints. Furthermore, we show how it can be used to efficiently support more complex functions such as calculating the occluded parts of space and accumulating the output from a semantic segmentation network.

show abstract

“…Finally, we validate the effectiveness of PointPillars enhancement with our MA mechanism in terms of cross-task generalization. This work is an extension of our conference paper [20], which has been extended with the novel MA mechanism design, a detailed description of the proposed PillarSegNet backbone model, along with an extended set of experiments on multiple datasets. In summary, the main contributions are:…”

Section: Introductionmentioning

confidence: 99%

MASS: Multi-Attentional Semantic Segmentation of LiDAR Data for Dense Top-View Understanding

Peng¹,

Fei²,

Yang³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

At the heart of all automated driving systems is the ability to sense the surroundings, e.g., through semantic segmentation of LiDAR sequences, which experienced a remarkable progress due to the release of large datasets such as SemanticKITTI and nuScenes-LidarSeg. While most previous works focus on sparse segmentation of the LiDAR input, dense output masks provide self-driving cars with almost complete environment information. In this paper, we introduce MASSa Multi-Attentional Semantic Segmentation model specifically built for dense top-view understanding of the driving scenes. Our framework operates on pillar-and occupancy features and comprises three attention-based building blocks: (1) a keypoint-driven graph attention, (2) an LSTM-based attention computed from a vector embedding of the spatial input, and (3) a pillar-based attention, resulting in a dense 360 • segmentation mask. With extensive experiments on both, SemanticKITTI and nuScenes-LidarSeg, we quantitatively demonstrate the effectiveness of our model, outperforming the state of the art by 19.0% on SemanticKITTI and reaching 32.7% in mIoU on nuScenes-LidarSeg, where MASS is the first work addressing the dense segmentation task. Furthermore, our multi-attention model is shown to be very effective for 3D object detection validated on the KITTI-3D dataset, showcasing its high generalizability to other tasks related to 3D vision.

show abstract

PillarSegNet: Pillar-based Semantic Grid Map Estimation using Sparse LiDAR Data

Cited by 29 publications

References 17 publications

A Multi-Task Recurrent Neural Network for End-to-End Dynamic Occupancy Grid Mapping

A Multi-Task Recurrent Neural Network for End-to-End Dynamic Occupancy Grid Mapping

Semantic 3D Grid Maps for Autonomous Driving

MASS: Multi-Attentional Semantic Segmentation of LiDAR Data for Dense Top-View Understanding

Contact Info

Product

Resources

About