Cali-sketch: Stroke calibration and completion for high-quality face image generation from human-like sketches

Xia, Weihao; Yang, Yujiu; Xue, Jing-Hao

doi:10.1016/j.neucom.2021.07.029

Cited by 18 publications

(5 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Second is deep-network aided loss-function definition. When training neural network models, instead of defining the loss function by directly using the difference between the generated and target sketches, we may input both the generated and target sketches into a deep network N , and define the loss function using the difference between the activities of N in response to the two inputs [79, 80, 81]. The blurring method introduced in this paper has its unique contribution compared to both approaches above.…”

Section: Discussionmentioning

confidence: 99%

Top-down generation of low-resolution representations improves visual perception and imagination

2021

Preprint

View full text Add to dashboard Cite

According to analysis-by-synthesis theories of perception, the primary visual cortex (V1) reconstructs visual stimuli through top-down pathway, and higher-order cortex reconstructs V1 activity. Experiments also found that neural representations are generated in a top-down cascade during visual imagination. What code does V1 provide higher-order cortex to reconstruct or simulate to improve perception or imaginative creativity? What unsupervised learning principles shape V1 for reconstructing stimuli so that V1 activity eigenspectrum is power-law with close-to-1 exponent? Using computational models, we reveal that reconstructing the activities of V1 complex cells facilitate higher-order cortex to form representations smooth to shape morphing of stimuli, improving perception and creativity. Power-law eigenspectrum with close-to-1 exponent results from the constraints of sparseness and temporal slowness when V1 is reconstructing stimuli, at a sparseness strength that best whitens V1 code and makes the exponent most insensitive to slowness strength. Our results provide fresh insights into V1 computation.

show abstract

Section: Discussionmentioning

confidence: 99%

Top-down generation of low-resolution representations improves visual perception and imagination

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Visual balance [14] Objective similarity SSIM [92], [129] [47], [93], [130], [17], [19], [95], [131] [38] PSNR [92] [47], [93], [130], [95], [131] [15]…”

Section: Characteristicmentioning

confidence: 99%

Evaluation Metrics for Intelligent Generation of Graphical Game Assets: A Systematic Survey-Based Framework

Fukaya,

Daylamani-Zad,

Agius

2024

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Generative systems for graphical assets have the potential to provide users with high quality assets at the push of a button. However, there are many forms of assets, and many approaches for producing them. Quantitative evaluation of these methods is necessary if practitioners wish to validate or compare their implementations. Furthermore, providing benchmarks for new methods to strive for or surpass. While most methods are validated using tried-and-tested metrics within their own domains, there is no unified method of finding the most appropriate. We present a framework based on a literature pool of close to 200 papers, that provides guidance in selecting metrics to evaluate the validity and quality of artefacts produced, and the operational capabilities of the method.

show abstract

“…Jo et al [23] propose a face editing method where users can edit face images using sketch and color. Both sketch-based and label-based categories put forward high requirements for the user's drawing [28]. It is challenging to synthesize natural and realistic images from poorly-drawn sketches or labels.…”

Section: Related Workmentioning

confidence: 99%

Towards Open-World Text-Guided Face Image Generation and Manipulation

Xia,

Yang,

Xue

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

The existing text-guided image synthesis methods can only produce limited quality results with at most 256 2 resolution and the textual instructions are constrained in a small Corpus. In this work, we propose a unified framework for both face image generation and manipulation that produces diverse and high-quality images with an unprecedented resolution at 1024 2 from multimodal inputs. More importantly, our method supports open-world scenarios, including both image and text, without any re-training, fine-tuning, or post-processing. To be specific, we propose a brand new paradigm of text-guided image generation and manipulation based on the superior characteristics of a pretrained GAN model. Our proposed paradigm includes two novel strategies. The first strategy is to train a text encoder to obtain latent codes that align with the hierarchically semantic of the aforementioned pretrained GAN model. The second strategy is to directly optimize the latent codes in the latent space of the pretrained GAN model with guidance from a pretrained language model. The latent codes can be randomly sampled from a prior distribution or inverted from a given image, which provides inherent supports for both image generation and manipulation from multi-modal inputs, such as sketches or semantic labels, with textual guidance. To facilitate text-guided multi-modal synthesis, we propose the MULTI-MODAL CELEBA-HQ, a large-scale dataset consisting of real face images and corresponding semantic segmentation map, sketch, and textual descriptions. Extensive experiments on the introduced dataset demonstrate the superior performance of our proposed method.

show abstract

Cali-sketch: Stroke calibration and completion for high-quality face image generation from human-like sketches

Cited by 18 publications

References 61 publications

Top-down generation of low-resolution representations improves visual perception and imagination

Top-down generation of low-resolution representations improves visual perception and imagination

Evaluation Metrics for Intelligent Generation of Graphical Game Assets: A Systematic Survey-Based Framework

Towards Open-World Text-Guided Face Image Generation and Manipulation

Contact Info

Product

Resources

About