Quantifying Point Cloud Realism Through Adversarially Learned Latent Representations

Triess, Larissa T.; Peter, David; Baur, Stefan A.; Zöllner, J. Marius

doi:10.1007/978-3-030-92659-5_44

Cited by 4 publications

(2 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In a recent breakthrough, Ding et al (2021Ding et al ( , 2023b introduced the pioneering model for this purpose, termed Continuous Conditional Generative Adversarial Networks (CcGANs), showcasing their superiority over conventional conditional GANs across various regression datasets. CcGANs have a wide spectrum of practical applications, including engineering inverse design (Heyrani Nobari, Chen, and Ahmed 2021; Fang, Shen, and Wang 2023), remote sensing image analysis (Giry-Fouquet et al 2022), model compression (Ding et al 2023a;Shi et al 2023), point cloud generation (Triess et al 2022), carbon sequestration (Stepien et al 2023), datadriven solutions for poroelasticity (Kadeethum et al 2022), etc. However, it's important to note that while CcGANs have shown success in these tasks, challenges remain when dealing with extremely sparse or imbalanced training data, leaving ample room to improve CcGAN models further.…”

Section: Introductionmentioning

confidence: 99%

Turning Waste into Wealth: Leveraging Low-Quality Samples for Enhancing Continuous Conditional Generative Adversarial Networks

Ding,

Wang,

2024

AAAI

View full text Add to dashboard Cite

Continuous Conditional Generative Adversarial Networks (CcGANs) enable generative modeling conditional on continuous scalar variables (termed regression labels). However, they can produce subpar fake images due to limited training data. Although Negative Data Augmentation (NDA) effectively enhances unconditional and class-conditional GANs by introducing anomalies into real training images, guiding the GANs away from low-quality outputs, its impact on CcGANs is limited, as it fails to replicate negative samples that may occur during the CcGAN sampling. We present a novel NDA approach called Dual-NDA specifically tailored for CcGANs to address this problem. Dual-NDA employs two types of negative samples: visually unrealistic images generated from a pre-trained CcGAN and label-inconsistent images created by manipulating real images' labels. Leveraging these negative samples, we introduce a novel discriminator objective alongside a modified CcGAN training algorithm. Empirical analysis on UTKFace and Steering Angle reveals that Dual-NDA consistently enhances the visual fidelity and label consistency of fake images generated by CcGANs, exhibiting a substantial performance gain over the vanilla NDA. Moreover, by applying Dual-NDA, CcGANs demonstrate a remarkable advancement beyond the capabilities of state-of-the-art conditional GANs and diffusion models, establishing a new pinnacle of performance. Our codes can be found at https://github.com/UBCDingXin/Dual-NDA.

show abstract

Section: Introductionmentioning

confidence: 99%

Turning Waste into Wealth: Leveraging Low-Quality Samples for Enhancing Continuous Conditional Generative Adversarial Networks

Ding,

Wang,

2024

AAAI

View full text Add to dashboard Cite

show abstract

“…CcGANs have been extensively applied across various domains requiring precise control over generative modeling of highdimensional data. These applications comprise engineering inverse design [2]- [4], data augmentation for hyperspectral imaging [5], remote sensing image processing [6], model compression [7], controllable point cloud generation [8], carbon sequestration [9], data-driven solutions for poroelasticity [10], and more. However, as reported in [11], the training process of CcGANs remains susceptible to extremely sparse or imbalanced training data due to the unstable adversarial mechanism, resulting in suboptimal outcomes.…”

Section: Introductionmentioning

confidence: 99%

Image Generation Using Continuous Conditional Generative Adversarial Networks

Ding

Wang

et al. 2022

Intelligent Systems Reference Library

View full text Add to dashboard Cite

Continuous Conditional Generative Modeling (CCGM) aims to estimate the distribution of high-dimensional data, typically images, conditioned on scalar continuous variables known as regression labels. While Continuous conditional Generative Adversarial Networks (CcGANs) were initially designed for this task, their adversarial training mechanism remains vulnerable to extremely sparse or imbalanced data, resulting in suboptimal outcomes. To enhance the quality of generated images, a promising alternative is to replace CcGANs with Conditional Diffusion Models (CDMs), renowned for their stable training process and ability to produce more realistic images. However, existing CDMs encounter challenges when applied to CCGM tasks due to several limitations such as inadequate U-Net architectures and deficient model fitting mechanisms for handling regression labels. In this paper, we introduce Continuous Conditional Diffusion Models (CCDMs), the first CDM designed specifically for the CCGM task. CCDMs address the limitations of existing CDMs by introducing specially designed conditional diffusion processes, a modified denoising U-Net with a custom-made conditioning mechanism, a novel hard vicinal loss for model fitting, and an efficient conditional sampling procedure. With comprehensive experiments on four datasets with varying resolutions ranging from 64 × 64 to 192 × 192, we demonstrate the superiority of the proposed CCDM over state-of-the-art CCGM models, establishing new benchmarks in CCGM. Extensive ablation studies validate the model design and implementation configuration of the proposed CCDM. Our code is publicly available at https://github.com/UBCDingXin/CCDM.

show abstract

A Realism Metric for Generated LiDAR Point Clouds

et al. 2022

View full text Add to dashboard Cite

A considerable amount of research is concerned with the generation of realistic sensor data. LiDAR point clouds are generated by complex simulations or learned generative models. The generated data is usually exploited to enable or improve downstream perception algorithms. Two major questions arise from these procedures: First, how to evaluate the realism of the generated data? Second, does more realistic data also lead to better perception performance? This paper addresses both questions and presents a novel metric to quantify the realism of LiDAR point clouds. Relevant features are learned from real-world and synthetic point clouds by training on a proxy classification task. In a series of experiments, we demonstrate the application of our metric to determine the realism of generated LiDAR data and compare the realism estimation of our metric to the performance of a segmentation model. We confirm that our metric provides an indication for the downstream segmentation performance.

show abstract

Quantifying Point Cloud Realism Through Adversarially Learned Latent Representations

Cited by 4 publications

References 20 publications

Turning Waste into Wealth: Leveraging Low-Quality Samples for Enhancing Continuous Conditional Generative Adversarial Networks

Turning Waste into Wealth: Leveraging Low-Quality Samples for Enhancing Continuous Conditional Generative Adversarial Networks

Image Generation Using Continuous Conditional Generative Adversarial Networks

A Realism Metric for Generated LiDAR Point Clouds

Contact Info

Product

Resources

About