L2R GAN: LiDAR-to-Radar Translation

Wang, Leichen; Goldluecke, Bastian; Anklam, Carsten

doi:10.1007/978-3-030-69535-4_43

Cited by 11 publications

(12 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Kernel-based Generative Learning The proposed method falls in the scope of transition-kernel-based generative learning [56,51,65,55,10,35]; specifically, it belongs to score-based generative learning [24,23,61,27]. The score-based generative models show comparable data modeling performance to those generative adversarial methods [33,5,68,62,59,36,8,4,28].…”

Section: Related Workmentioning

confidence: 99%

“…Many early mapping-based modality translations methods rely on pixel-level modeling between the source and target modality. Generative adversarial network (GAN) [33,5,68,62,59,36,4] based methods were proposed due to various shortcomings of the previous mapping-based approaches. The generative model is usually used for modeling the target modality directly, thus achieving translation realism [1,69,8].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Zero-shot-Learning Cross-Modality Data Translation Through Mutual Information Guided Stochastic Diffusion

Wang¹,

Yingyu²,

Sermesant³

et al. 2023

Preprint

View full text Add to dashboard Cite

Cross-modality data translation has attracted great interest in image computing. Deep generative models (e.g., GANs) show performance improvement in tackling those problems. Nevertheless, as a fundamental challenge in image translation, the problem of Zero-shot-Learning Cross-Modality Data Translation with fidelity remains unanswered. This paper proposes a new unsupervised zero-shot-learning method named Mutual Information guided Diffusion cross-modality data translation Model (MIDiffusion), which learns to translate the unseen source data to the target domain. The MIDiffusion leverages a score-matching-based generative model, which learns the prior knowledge in the target domain. We propose a differentiable local-wise-MI-Layer (LM I) for conditioning the iterative denoising sampling. The LM I captures the identical cross-modality features in the statistical domain for the diffusion guidance; thus, our method does not require retraining when the source domain is changed, as it does not rely on any direct mapping between the source and target domains. This advantage is critical for applying cross-modality data translation methods in practice, as a reasonable amount of source domain dataset is not always available for supervised training. We empirically show the advanced performance of MIDiffusion in comparison with an influential group of generative models, including adversarial-based and other scorematching-based models.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Zero-shot-Learning Cross-Modality Data Translation Through Mutual Information Guided Stochastic Diffusion

Wang¹,

Yingyu²,

Sermesant³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…The decoder generates a radar intensity map in the polar grid conditioned on the encoded feature and random noise. Generative models can also be used in cross-modality data generation, for example GAN-based LiDAR-to-radar generation [104], GAN-based radar-to-image generation [105], and VAE-based radar-to-image generation [106].…”

Section: Synthetic Datamentioning

confidence: 99%

Towards Deep Radar Perception for Autonomous Driving: Datasets, Methods, and Challenges

Zhou

Liu

Zhao

et al. 2022

Sensors

View full text Add to dashboard Cite

With recent developments, the performance of automotive radar has improved significantly. The next generation of 4D radar can achieve imaging capability in the form of high-resolution point clouds. In this context, we believe that the era of deep learning for radar perception has arrived. However, studies on radar deep learning are spread across different tasks, and a holistic overview is lacking. This review paper attempts to provide a big picture of the deep radar perception stack, including signal processing, datasets, labelling, data augmentation, and downstream tasks such as depth and velocity estimation, object detection, and sensor fusion. For these tasks, we focus on explaining how the network structure is adapted to radar domain knowledge. In particular, we summarise three overlooked challenges in deep radar perception, including multi-path effects, uncertainty problems, and adverse weather effects, and present some attempts to solve them.

show abstract

“…However, it cannot formulate the optimal LiDAR input format and also cannot handle weather changes, such as rain and fog. Additionally, there is the LiDAR translation research between LiDAR and radar for data augmentation [ 36 ]. PU-GAN [ 37 ] learns a rich variety of point distributions from the latent space and up-samples points by constructing an up–down–up expansion unit in the generator.…”

Section: Related Workmentioning

confidence: 99%

GAN-Based LiDAR Translation between Sunny and Adverse Weather for Autonomous Driving and Driving Simulation

Lee

Shiotsuka

Nishimori

et al. 2022

Sensors

View full text Add to dashboard Cite

Autonomous driving requires robust and highly accurate perception technologies. Various deep learning algorithms based on only image processing satisfy this requirement, but few such algorithms are based on LiDAR. However, images are only one part of the perceptible sensors in an autonomous driving vehicle; LiDAR is also essential for the recognition of driving environments. The main reason why there exist few deep learning algorithms based on LiDAR is a lack of data. Recent translation technology using generative adversarial networks (GANs) has been proposed to deal with this problem. However, these technologies focus on only image-to-image translation, although a lack of data occurs more often with LiDAR than with images. LiDAR translation technology is required not only for data augmentation, but also for driving simulation, which allows algorithms to practice driving as if they were commanding a real vehicle, before doing so in the real world. In other words, driving simulation is a key technology for evaluating and verifying algorithms which are practically applied to vehicles. In this paper, we propose a GAN-based LiDAR translation algorithm for autonomous driving and driving simulation. It is the first LiDAR translation approach that can deal with various types of weather that are based on an empirical approach. We tested the proposed method on the JARI data set, which was collected under various adverse weather scenarios with diverse precipitation and visible distance settings. The proposed method was also applied to the real-world Spain data set. Our experimental results demonstrate that the proposed method can generate realistic LiDAR data under adverse weather conditions.

show abstract

L2R GAN: LiDAR-to-Radar Translation

Cited by 11 publications

References 36 publications

Zero-shot-Learning Cross-Modality Data Translation Through Mutual Information Guided Stochastic Diffusion

Zero-shot-Learning Cross-Modality Data Translation Through Mutual Information Guided Stochastic Diffusion

Towards Deep Radar Perception for Autonomous Driving: Datasets, Methods, and Challenges

GAN-Based LiDAR Translation between Sunny and Adverse Weather for Autonomous Driving and Driving Simulation

Contact Info

Product

Resources

About