DSCnet: Replicating Lidar Point Clouds With Deep Sensor Cloning

Tomasello, Paden; Sidhu, Sammy; Shen, Anting; Moskewicz, Matthew W.; Redmon, Nobie; Joshi, Gayatri; Phadte, Romi; Jain, Paras; Iandola, Forrest

doi:10.1109/cvprw.2019.00171

Cited by 4 publications

(5 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Concurrent with our work, Tomasello et al [15] explore conditional lidar synthesis from RGB images. The authors use the same 2D spherical mapping proposed in [6].…”

Section: B Grid-based Lidar Generationmentioning

confidence: 84%

Deep Generative Modeling of LiDAR Data

Caccia

Hoof

Courville

et al. 2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

Building models capable of generating structured output is a key challenge for AI and robotics. While generative models have been explored on many types of data, little work has been done on synthesizing lidar scans, which play a key role in robot mapping and localization. In this work, we show that one can adapt deep generative models for this task by unravelling lidar scans into a 2D point map. Our approach can generate high quality samples, while simultaneously learning a meaningful latent representation of the data. We demonstrate significant improvements against state-of-the-art point cloud generation methods. Furthermore, we propose a novel data representation that augments the 2D signal with absolute positional information. We show that this helps robustness to noisy and imputed input; the learned model can recover the underlying lidar scan from seemingly uninformative data.

show abstract

“…Concurrent with our work, Tomasello et al [15] explore conditional lidar synthesis from RGB images. The authors use the same 2D spherical mapping proposed in [6].…”

Section: B Grid-based Lidar Generationmentioning

confidence: 84%

Deep Generative Modeling of LiDAR Data

Caccia

Hoof

Courville

et al. 2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

show abstract

“…To fill this gap, the LiDAR matching-based localization [3,4,8] with the prior map is a promising solution to provide the globally referenced and accurate positioning. Currently, the existing LiDAR map matching-based localization solution mainly relies on the 360 • rotating mechanical LiDAR, such as the 64 channels Velodyne HDL 64(about US$75,000) [9]. Specifically, the map matchingbased localization is achieved by associating the real-time 3D point clouds from LiDAR with the globally referenced prior map.…”

Section: Introductionmentioning

confidence: 99%

Low‐cost solid‐state LiDAR/inertial‐based localization with prior map for autonomous systems in urban scenarios

Zhong

Huang

Zhang

et al. 2022

IET Intelligent Trans Sys

View full text Add to dashboard Cite

A low‐cost and accurate positioning solution is significant for the massive deployment of fully autonomous driving vehicles (ADV). Conventional mechanical LiDAR has proven its performance, but its high cost hinders the massive production of autonomous vehicles. This paper proposes a low‐cost LiDAR/inertial‐based localization solution for autonomous systems with prior maps in urban areas. Instead of relying on the costly mechanical LiDAR, this paper proposes to utilize the solid‐state LiDAR (SSL) with the prior map to estimate the position of the vehicle by matching the real‐time point clouds from the SSL and the prior map using the normal distribution transformation (NDT) algorithm. However, the field of view (FOV) of the SSL is signifcantly smaller than the conventional mechanical LiDAR, which can easily lead to failure during the NDT map matching. To fill this gap, this paper proposes to exploit the complementariness of the inertial measurement unit (IMU) and the SSL, where the IMU pre‐integration provides a coarse but high‐frequency initial guess to the map matching. To evaluate the effectiveness of the proposed method in this paper, the authors collect the dataset in two typical urban scenarios through a pedestrian hand‐hold and a vehicle driving condition. The results reveal that the SSL‐only‐based localization is significantly challenged in dynamic scenarios. With the help of the IMU, the robustness of the proposed method is significantly improved, achieving an accuracy of within 0.5 m. To show the sensitivity of the SSL‐based map matching against the initial guess of the state, this paper also presents the convergence results of the map matching under different levels of accuracy in terms of the initial guess.

show abstract

“…As a product of these public datasets, we have seen an explosion of DNN model development in literature (discussed in section 2), where we are seeing the bar being raised year over year on these tasks. However we often see that these cutting edge models usually only operate on one modality and one task such as in [21,26] or that they perform multi-modal sensor fusion in a way that the sensor inputs are entangled and non-modular [3,18,22,27,30].…”

Section: Introductionmentioning

confidence: 99%

Scalable Primitives for Generalized Sensor Fusion in Autonomous Vehicles

Sidhu¹,

Wang²,

Naseer³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

In autonomous driving, there has been an explosion in the use of deep neural networks for perception, prediction and planning tasks. As autonomous vehicles (AVs) move closer to production, multi-modal sensor inputs and heterogeneous vehicle fleets with different sets of sensor platforms are becoming increasingly common in the industry. However, neural network architectures typically target specific sensor platforms and are not robust to changes in input, making the problem of scaling and model deployment particularly difficult. Furthermore, most players still treat the problem of optimizing software and hardware as entirely independent problems. We propose a new end to end architecture, Generalized Sensor Fusion (GSF), which is designed in such a way that both sensor inputs and target tasks are modular and modifiable. This enables AV system designers to easily experiment with different sensor configurations and methods and opens up the ability to deploy on heterogeneous fleets using the same models that are shared across a large engineering organization. Using this system, we report experimental results where we demonstrate near-parity of an expensive high-density (HD) LiDAR sensor with a cheap low-density (LD) LiDAR plus camera setup in the 3D object detection task. This paves the way for the industry to jointly design hardware and software architectures as well as large fleets with heterogeneous configurations.

show abstract

DSCnet: Replicating Lidar Point Clouds With Deep Sensor Cloning

Cited by 4 publications

References 24 publications

Deep Generative Modeling of LiDAR Data

Deep Generative Modeling of LiDAR Data

Low‐cost solid‐state LiDAR/inertial‐based localization with prior map for autonomous systems in urban scenarios

Scalable Primitives for Generalized Sensor Fusion in Autonomous Vehicles

Contact Info

Product

Resources

About