NaturalInversion: Data-Free Image Synthesis Improving Real-World Consistency

Abstract

We introduce NaturalInversion, a novel model inversion-based method to synthesize images that agrees well with the original data distribution without using real data. In NaturalInversion, we propose: (1) a Feature Transfer Pyramid which uses enhanced image prior of the original data by combining the multi-scale feature maps extracted from the pre-trained classifier, (2) a one-to-one approach generative model where only one batch of images are synthesized by one generator to bring the non-linearity to optimization and to ease the overall optimizing process, (3) learnable Adaptive Channel Scaling parameters which are end-to-end trained to scale the output image channel to utilize the original image prior further. With our NaturalInversion, we synthesize images from classifiers trained on CIFAR-10/100 and show that our images are more consistent with original data distribution than prior works by visualization and additional analysis. Furthermore, our synthesized images outperform prior works on various applications such as knowledge distillation and pruning, demonstrating the effectiveness of our proposed method.

State of the Art on Diffusion Models for Visual Computing

“…Different motion representations have been used in prior work without a clear advantage of one over the other. While all assume an underlying body model, some use a minimal representation such as the 6D joint rotations [TCL23, KKC23], while others allow a redundant representation by having both positions and rotations [TRG*23, CJL*23]. It is also a common practice to include contact information as part of the motion representation to address concerns like foot sliding or surface penetration [TCL23].…”

Music conditioned 2D hand gesture dance generation with HGS

“…In addition, Zhuang et al 16 and Aristidou et al 17 used the dance posture melody curve to generate dance posture, but the actual effect is less satisfactory. There is also work focused on text2motion, [18][19][20][21][22] which takes a similar solution to the dance pose generation work. These work 23 focuses on the relationship between words and actions.…”

PoseScript: 3D Human Poses from Natural Language

“…3D human pose generation. Previous works have mainly focused on the generation of pose sequences, conditioning on music [30,31], context [10], past poses [63,64], text labels [20,46] and mostly on text descriptions [32,1,61,2,17,47,18,56,19,27]. Some works push it one step further and also attempt to synthesize the mesh appearance [24,62], leveraging large pretrained models like CLIP [50].…”