HyperNST: Hyper-Networks for Neural Style Transfer

Ruta, Dan; Gilbert, Andrew; Motiian, Saeid; Faieta, Baldo; Lin, Zhe; Collomosse, John

doi:10.1007/978-3-031-25056-9_14

Cited by 5 publications

(2 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Generalized one‐shot adaptation [ZLH*22] focuses on preserving decorations in a new domain. HyperNST [RGM*23] defines one‐shot face stylization as a style transfer problem.…”

Section: Related Workmentioning

confidence: 99%

Multi‐Modal Face Stylization with a Generative Prior

Li,

Dong,

Lin

et al. 2023

Computer Graphics Forum

View full text Add to dashboard Cite

In this work, we introduce a new approach for face stylization. Despite existing methods achieving impressive results in this task, there is still room for improvement in generating high‐quality artistic faces with diverse styles and accurate facial reconstruction. Our proposed framework, MMFS, supports multi‐modal face stylization by leveraging the strengths of StyleGAN and integrates it into an encoder‐decoder architecture. Specifically, we use the mid‐resolution and high‐resolution layers of StyleGAN as the decoder to generate high‐quality faces, while aligning its low‐resolution layer with the encoder to extract and preserve input facial details. We also introduce a two‐stage training strategy, where we train the encoder in the first stage to align the feature maps with StyleGAN and enable a faithful reconstruction of input faces. In the second stage, the entire network is fine‐tuned with artistic data for stylized face generation. To enable the fine‐tuned model to be applied in zero‐shot and one‐shot stylization tasks, we train an additional mapping network from the large‐scale Contrastive‐Language‐Image‐Pre‐training (CLIP) space to a latent w+ space of fine‐tuned StyleGAN. Qualitative and quantitative experiments show that our framework achieves superior performance in both one‐shot and zero‐shot face stylization tasks, outperforming state‐of‐the‐art methods by a large margin.

show abstract

“…Generalized one‐shot adaptation [ZLH*22] focuses on preserving decorations in a new domain. HyperNST [RGM*23] defines one‐shot face stylization as a style transfer problem.…”

Section: Related Workmentioning

confidence: 99%

Multi‐Modal Face Stylization with a Generative Prior

Li,

Dong,

Lin

et al. 2023

Computer Graphics Forum

View full text Add to dashboard Cite

show abstract

“…[9] propose a lightweight network that can simultaneously achieve color transfer and texture transfer, achieving state-of-the-art results in both tasks. [20], [1], [18] employed diffusion for image style transfer, offering a more diverse range of transfers that can not only replace textures but also reasonably modify the image content. However, diffusionbased methods entail significantly higher computational costs compared to other types of algorithms.…”

Section: Introductionmentioning

confidence: 99%

Neural Style Transfer for 3d Meshes

Kang

Xiao

Cao

et al. 2023

Preprint

View full text Add to dashboard Cite

A brief review of hypernetworks in deep learning

Chauhan,

Zhou,

et al. 2024

Artif Intell Rev

View full text Add to dashboard Cite

Hypernetworks, or hypernets for short, are neural networks that generate weights for another neural network, known as the target network. They have emerged as a powerful deep learning technique that allows for greater flexibility, adaptability, dynamism, faster training, information sharing, and model compression. Hypernets have shown promising results in a variety of deep learning problems, including continual learning, causal inference, transfer learning, weight pruning, uncertainty quantification, zero-shot learning, natural language processing, and reinforcement learning. Despite their success across different problem settings, there is currently no comprehensive review available to inform researchers about the latest developments and to assist in utilizing hypernets. To fill this gap, we review the progress in hypernets. We present an illustrative example of training deep neural networks using hypernets and propose categorizing hypernets based on five design criteria: inputs, outputs, variability of inputs and outputs, and the architecture of hypernets. We also review applications of hypernets across different deep learning problem settings, followed by a discussion of general scenarios where hypernets can be effectively employed. Finally, we discuss the challenges and future directions that remain underexplored in the field of hypernets. We believe that hypernetworks have the potential to revolutionize the field of deep learning. They offer a new way to design and train neural networks, and they have the potential to improve the performance of deep learning models on a variety of tasks. Through this review, we aim to inspire further advancements in deep learning through hypernetworks.

show abstract

HyperNST: Hyper-Networks for Neural Style Transfer

Cited by 5 publications

References 15 publications

Multi‐Modal Face Stylization with a Generative Prior

Multi‐Modal Face Stylization with a Generative Prior

Neural Style Transfer for 3d Meshes

A brief review of hypernetworks in deep learning

Contact Info

Product

Resources

About