Draw Your Art Dream: Diverse Digital Art Synthesis with Multimodal Guided Diffusion

Nisha, Huang,; Tang, Fan; Dong, Weiming; Xu, Changsheng

doi:10.1145/3503161.3548282

Cited by 21 publications

(7 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…2(b), these methods still face the challenge of generating images with styles that are inconsistent with textual prompts. Our research closely follows the previous work [1,4,27,28], focusing on converting multimodal prompts into realistic artistic images and achieving innovations in reconstructing and editing existing images.…”

Section: Related Workmentioning

confidence: 78%

School Wireless Network Classroom Teaching System Based on Artificial Intelligence

Zhang

2023

Applied Artificial Intelligence

View full text Add to dashboard Cite

https://anonymous.4open.science/r/CreativeSynth-52A8 Figure 1. Our CreativeSynth unified framework is capable of generating personalized digital art when supplied with an art image, drawing on prompts from either unimodal or multimodal prompts. This methodology not only yields artwork with high-fidelity realism but also effectively upholds the foundational concepts, composition, stylistic elements, and visual symbolism intrinsic to genuine artworks. Cre-ativeSynth supports a wide array of intriguing applications, including (a) image variation, (b) image editing, (c) style transfer, (d) image fusion, and (e) multimodal blending.

show abstract

Section: Related Workmentioning

confidence: 78%

School Wireless Network Classroom Teaching System Based on Artificial Intelligence

Zhang

2023

Applied Artificial Intelligence

View full text Add to dashboard Cite

show abstract

“…These models can generate images that are closely aligned with the input text prompt. Motivated by these successes, many works attempt to utilize pre-trained T2I diffusion models for various tasks such as text-driven image editing [2,13,16,28,29,41]. However, using 2D diffusion model to achieve fine-grained text-driven 3D stylization is seldom explored and remains an open problem in the multimedia and vision field.…”

Section: Related Workmentioning

confidence: 99%

3DStyle-Diffusion: Pursuing Fine-grained Text-driven 3D Stylization with 2D Diffusion Models

Yang,

Chen,

Pan

et al. 2023

Proceedings of the 31st ACM International Conference on Multimedia

View full text Add to dashboard Cite

3D content creation via text-driven stylization has played a fundamental challenge to multimedia and graphics community. Recent advances of cross-modal foundation models (e.g., CLIP) have made this problem feasible. Those approaches commonly leverage CLIP to align the holistic semantics of stylized mesh with the given text prompt. Nevertheless, it is not trivial to enable more controllable stylization of fine-grained details in 3D meshes solely based on such semantic-level cross-modal supervision. In this work, we propose a

show abstract

“…One of the large-scale multimodal pre-training models that have received many applications is the Contrastive Language Image Pretraining (CLIP) [38] model, which was pre-trained on 400 million text-image samples. In parallel, various new image synthesis methods [15,18,25,27,37,39] have highlighted the richness of the vast visual and linguistic realm encompassed by CLIP. Nonetheless, manipulating existing items in arbitrary, actual pictures continues to remain tricky.…”

Section: Introductionmentioning

confidence: 99%

Region-Aware Diffusion for Zero-shot Text-driven Image Editing

Nisha¹,

Tang²,

Dong³

et al. 2023

Preprint

View full text Add to dashboard Cite

Fig. 1. The results of the proposed region-aware diffusion model (RDM). The texts adhere to the phrase rule "A → B", indicating that RDM transforms entity A into entity B.Image manipulation under the guidance of textual descriptions has recently received a broad range of attention. In this study, we focus on the regional editing of images with the guidance of given text prompts. Different from current mask-based image editing methods, we propose a novel region-aware diffusion model (RDM) for entity-level image editing, which could automatically locate the region of interest and replace it following given text prompts. To strike a balance between image fidelity and inference speed, we design the intensive diffusion pipeline by combing latent space diffusion and enhanced directional guidance. In addition, to preserve

show abstract

Draw Your Art Dream: Diverse Digital Art Synthesis with Multimodal Guided Diffusion

Abstract: Figure 1: Digital art paintings generated by the proposed multimodal guided artwork diffusion (MGAD) model.

Cited by 21 publications

References 56 publications

School Wireless Network Classroom Teaching System Based on Artificial Intelligence

School Wireless Network Classroom Teaching System Based on Artificial Intelligence

3DStyle-Diffusion: Pursuing Fine-grained Text-driven 3D Stylization with 2D Diffusion Models

Region-Aware Diffusion for Zero-shot Text-driven Image Editing

Contact Info

Product

Resources

About