Perception Prioritized Training of Diffusion Models

Choi, Joon-Seok; Lee, Jungbeom; Shin, Chaehun; Kim, Sung-Won; Kim, Hyun Woo; Yoon, Sungroh

doi:10.1109/cvpr52688.2022.01118

Cited by 98 publications

(26 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Fig. 1 shows the restoration results of real artifacts generated by GAN [17], GPT [8], DDPM model [3], respectively. DiffGAR exhibits better image restoration capabilities compared to other models in all three kinds of generative artifacts.…”

Section: Comparison Resultsmentioning

confidence: 99%

DiffGAR: Model-Agnostic Restoration from Generative Artifacts Using Image-to-Image Diffusion Models

Yin¹,

Huang²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Section: Comparison Resultsmentioning

confidence: 99%

DiffGAR: Model-Agnostic Restoration from Generative Artifacts Using Image-to-Image Diffusion Models

Yin¹,

Huang²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

“…It has been observed in [1,5,6] that low-frequency information, i.e., coarse features such as pose and facial shape are learned in the earlier timesteps (e.g., 0 < SNR(t) < 10 −2 ) while high-frequency information such as fine-grained features and imperceptible details are encoded in later timesteps (e.g., 10 0 < SNR(t) < 10 4 ) in the reverse diffusion process. Here, SNR(t) stands for signal-to-noise ratio per timestep [26].…”

Section: Timestep Scheduling For Conditioningmentioning

confidence: 88%

“…However, their approach requires costly optimization during inference and doesn't support controlling the final generation. Inductive Bias of Diffusion Models: On top of the inductive bias [5,6] of Diffusion Models, eDiffi [1] proposed to train models specialized to a subset of the timesteps to improve generations drastically. Mag-icMix [33] interpolates noise maps while providing different embeddings at different timesteps.…”

Section: Inference Only Editingmentioning

confidence: 99%

“…In addition, we leverage the inductive bias [1,5,6] of diffusion models that learns low frequency layout information in earlier steps and high frequency or imperceptible details in the later steps of the reverse diffusion process, to further improve results. We use a predefined controllable timestep dependent weight schedule to compose the content and style codes during generation.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Towards Enhanced Controllability of Diffusion Models

Cho¹,

Ravi²,

Harikumar³

et al. 2023

Preprint

View full text Add to dashboard Cite

Figure 1. The proposed framework enhances editability in diffusion models by conditioning the generation on two latent spaces, i.e., content and style. The latent codes are effectively combined to generate novel images. Our proposed sampling technique and timestep scheduling further improve controllability. (a) Magnitude of style can be controlled to translate semantic information from the style image. (b) The learned style space supports smooth interpolations while (c) PCA on the learned latent space gives disentangled attribute specific manipulation directions. Details are provided in sections D.1 and D.2 in appendix. More results can be found in Fig. 25 in appendix.

show abstract

“…To tackle this data consistency problem and enforce the model to focus more on morphological patterns, they first feed input images into a color normalization module [124] to unify the domain of all images. In addition, they apply a morphology levels prioritization module [125] that designates higher weight values to the loss at earlier levels to emphasize perceptual information and lower weights to the loss at later levels, resulting in higher Figure 13: Visual comparison of DDM [52], VM [120], and VM-Diff [122] for generating temporal cardiac images. The deformed intermediate frames S( φ) (right) are constructed using the source and target, and produced the deformation fields φ (left).…”

Section: Image Generationmentioning

confidence: 99%

Diffusion Models for Medical Image Analysis: A Comprehensive Survey

Kazerouni¹,

Aghdam²,

Heidari³

et al. 2022

Preprint

View full text Add to dashboard Cite

Denoising diffusion models, a class of generative models, have garnered immense interest lately in various deep-learning problems. A diffusion probabilistic model defines a forward diffusion stage where the input data is gradually perturbed over several steps by adding Gaussian noise and then learns to reverse the diffusion process to retrieve the desired noise-free data from noisy data samples. Diffusion models are widely appreciated for their strong mode coverage and quality of the generated samples in spite of their known computational burdens. Capitalizing on the advances in computer vision, the field of medical imaging has also observed a growing interest in diffusion models. With the aim of helping the researcher navigate this profusion, this survey intends to provide a comprehensive overview of diffusion models in the discipline of medical image analysis. Specifically, we start with an introduction to the solid theoretical foundation and fundamental concepts behind diffusion models and the three generic diffusion modeling frameworks, namely, diffusion probabilistic models, noise-conditioned score networks, and stochastic differential equations. Then, we provide a systematic taxonomy of diffusion models in the medical domain and propose a multi-perspective categorization based on their application, imaging modality, organ of interest, and algorithms. To this end, we cover extensive applications of diffusion models in the medical domain, including segmentation, anomaly detection, image-to-image translation, 2/3D generation, reconstruction, denoising, and other medically-related challenges. Furthermore, we emphasize the practical use case of some selected approaches, and then we discuss the limitations of the diffusion models in the medical domain and propose several directions to fulfill the demands of this field. Finally, we gather the overviewed studies with their available open-source implementations at our GitHub 1 . We aim to update the relevant latest papers within it regularly.

show abstract

Perception Prioritized Training of Diffusion Models

Cited by 98 publications

References 11 publications

DiffGAR: Model-Agnostic Restoration from Generative Artifacts Using Image-to-Image Diffusion Models

DiffGAR: Model-Agnostic Restoration from Generative Artifacts Using Image-to-Image Diffusion Models

Towards Enhanced Controllability of Diffusion Models

Diffusion Models for Medical Image Analysis: A Comprehensive Survey

Contact Info

Product

Resources

About