TRACT: Denoising Diffusion Models with Transitive Closure Time-Distillation

Abstract

Denoising Diffusion models have demonstrated their proficiency for generative sampling. However, generating good samples often requires many iterations. Consequently, techniques such as binary time-distillation (BTD) have been proposed to reduce the number of network calls for a fixed architecture. In this paper, we introduce TRAnsitive Closure Time-distillation (TRACT), a new method that extends BTD. For single step diffusion, TRACT improves FID by up to 2.4× on the same architecture, and achieves new single-step Denoising Diffusion Implicit Models (DDIM) state-of-the-art FID (7.4 for ImageNet64, 3.8 for CIFAR10). Finally we tease apart the method through extended ablations. The PyTorch [37] implementation will be released soon.

CaloClouds II: ultra-fast geometry-independent highly-granular calorimeter simulation

“…Another research direction to speed up generative models is the distillation of diffusion models into models which require significantly fewer function evaluations during sampling than the original model [71][72][73][74][75]. Recently, consistency models have been introduced as a novel kind of generative model allowing for single and multi-step data generation [76].…”