Generating a Fusion Image: One's Identity and Another's Shape

Joo, Donggyu; Kim, Doyeon; Kim, Junmo

doi:10.1109/cvpr.2018.00176

Cited by 46 publications

(31 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, these approaches have limitations: for example, they rely on pre-trained models for extracting object representations that require costly ground-truth data annotations [9,47,3]. Furthermore, these works do not address the problem of animating arbitrary objects: instead, consid-ering a single object category [50] or learning to translate videos from one specific domain to another [4,25]. This paper addresses some of these limitations by introducing a novel deep learning framework for animating a static image using a driving video.…”

Section: Introductionmentioning

confidence: 99%

Animating Arbitrary Objects via Deep Motion Transfer

Siarohin

Lathuilière

Tulyakov

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

367

379

View full text Add to dashboard Cite

This paper introduces a novel deep learning framework for image animation. Given an input image with a target object and a driving video sequence depicting a moving object, our framework generates a video in which the target object is animated according to the driving sequence. This is achieved through a deep architecture that decouples appearance and motion information. Our framework consists of three main modules: (i) a Keypoint Detector unsupervisely trained to extract object keypoints, (ii) a Dense Motion prediction network for generating dense heatmaps from sparse keypoints, in order to better encode motion information and (iii) a Motion Transfer Network, which uses the motion heatmaps and appearance information extracted from the input image to synthesize the output frames. We demonstrate the effectiveness of our method on several benchmark datasets, spanning a wide variety of object appearances, and show that our approach outperforms stateof-the-art image animation and video generation methods. Our source code is publicly available 1 .

show abstract

Section: Introductionmentioning

confidence: 99%

Animating Arbitrary Objects via Deep Motion Transfer

Siarohin

Lathuilière

Tulyakov

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

367

379

View full text Add to dashboard Cite

show abstract

“…Similarly, we apply our representation of motion (pose stick figures) to different target subjects to generate new motions while in contrast our work specializes on synthesizing detailed dance movements. Modern approaches have shown success in generating detailed images of human subjects in novel poses [10,16,22,23,31]. Furthermore, recent methods can synthesize such images for temporally coherent video [2] and future prediction [31].…”

Section: Related Workmentioning

confidence: 99%

Everybody Dance Now

Chan

Ginosar

Zhou

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

673

596

View full text Add to dashboard Cite

“…StarGAN [23] is a GAN that learns the mappings among multiple domains using only a single generator and a discriminator, training effectively from images of all domains. FusionGAN [24] generates a fusion image with the identity of input image x and the shape of input image y. This network can simultaneously train on more than two image datasets in an unsupervised manner.…”

Section: Related Workmentioning

confidence: 99%

Game Sprite Generator Using a Multi Discriminator GAN

2019

KSII TIIS

View full text Add to dashboard Cite

This paper proposes an image generation method using a Multi Discriminator Generative Adversarial Net (MDGAN) as a next generation 2D game sprite creation technique. The proposed GAN is an Autoencoder-based model that receives three areas of information-color, shape, and animation, and combines them into new images. This model consists of two encoders that extract color and shape from each image, and a decoder that takes all the values of each encoder and generates an animated image. We also suggest an image processing technique during the learning process to remove the noise of the generated images. The resulting images show that 2D sprites in games can be generated by independently learning the three image attributes of shape, color, and animation. The proposed system can increase the productivity of massive 2D image modification work during the game development process. The experimental results demonstrate that our MDGAN can be used for 2D image sprite generation and modification work with little manual cost.

show abstract

Generating a Fusion Image: One's Identity and Another's Shape

Cited by 46 publications

References 20 publications

Animating Arbitrary Objects via Deep Motion Transfer

Animating Arbitrary Objects via Deep Motion Transfer

Everybody Dance Now

Game Sprite Generator Using a Multi Discriminator GAN

Contact Info

Product

Resources

About