Split and Match: Example-Based Adaptive Patch Sampling for Unsupervised Style Transfer

Frigo, Oriel; Sabater, Neus; Delon, Julie; Hellier, Pierre

doi:10.1109/cvpr.2016.66

Cited by 121 publications

(88 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, most of these NPR stylisation algorithms are designed for particular artistic styles [3], [4] and cannot be easily extended to other styles. In the community of computer vision, style transfer is usually studied as a generalised problem of texture synthesis, which is to extract and transfer the texture from the source to target [5], [6], [7], [8]. Hertzmann et al [9] further propose a framework named image analogies to perform a generalised style transfer by learning the analogous transformation from the provided example pairs of unstylised and stylised images.…”

Section: Introductionmentioning

confidence: 99%

Neural Style Transfer: A Review

Jing

Yang

Feng

et al. 2020

IEEE Trans. Visual. Comput. Graphics

592

310

View full text Add to dashboard Cite

The seminal work of Gatys et al. demonstrated the power of Convolutional Neural Networks (CNNs) in creating artistic imagery by separating and recombining image content and style. This process of using CNNs to render a content image in different styles is referred to as Neural Style Transfer (NST). Since then, NST has become a trending topic both in academic literature and industrial applications. It is receiving increasing attention and a variety of approaches are proposed to either improve or extend the original NST algorithm. In this paper, we aim to provide a comprehensive overview of the current progress towards NST. We first propose a taxonomy of current algorithms in the field of NST. Then, we present several evaluation methods and compare different NST algorithms both qualitatively and quantitatively. The review concludes with a discussion of various applications of NST and open problems for future research. A list of papers discussed in this review, corresponding codes, pre-trained models and more comparison results are publicly available at: https://github.com/ycjing/Neural-Style-Transfer-Papers. IndexTerms-Neural style transfer (NST), convolutional neural network ! Neural Style Transfer Example-Based Techniques Colour Image Analogy Texture Model-Optimisation-Based Offline Neural Methods Multiple-Style-Per-Model Neural Methods Dumoulin'17 [53] Chen'17 [54] Li'17 [55] Zhang'17 [56] Luan'17 [84] Mechrez'17 [85]Photorealistic Liao'17 [88] Attribute Champandard'16 [65] Doodle Ruder'16 [74] Video Selim'16 [73] Portrait Castillo'17 [71] Instance Gatys'17 [60] Improvement Image Gatys'16 [10] Li'17 [42] Risser'17 [44] Li'17 [45] Li'16 [46] Image Champandard'16 [65] Chen'16 [68] Mechrez'18 [69]

show abstract

Section: Introductionmentioning

confidence: 99%

Neural Style Transfer: A Review

Jing

Yang

Feng

et al. 2020

IEEE Trans. Visual. Comput. Graphics

592

310

View full text Add to dashboard Cite

show abstract

“…Earlier methods [18] are usually non-parametric and are build upon low-level image features. Inspired by Image Analogies [18], approaches [11,30,39,40] are based on finding dense correspondence between content and style image and often require image pairs to depict similar content. Therefore, these methods do not scale to the setting of arbitrary content images.…”

Section: Related Workmentioning

confidence: 99%

A Style-Aware Content Loss for Real-Time HD Style Transfer

Sanakoyeu

Kotovenko

Lang

et al. 2018

Lecture Notes in Computer Science

195

135

View full text Add to dashboard Cite

Recently, style transfer has received a lot of attention. While much of this research has aimed at speeding up processing, the approaches are still lacking from a principled, art historical standpoint: a style is more than just a single image or an artist, but previous work is limited to only a single instance of a style or shows no benefit from more images. Moreover, previous work has relied on a direct comparison of art in the domain of RGB images or on CNNs pre-trained on Ima-geNet, which requires millions of labeled object bounding boxes and can introduce an extra bias, since it has been assembled without artistic consideration. To circumvent these issues, we propose a style-aware content loss, which is trained jointly with a deep encoder-decoder network for real-time, high-resolution stylization of images and videos. We propose a quantitative measure for evaluating the quality of a stylized image and also have art historians rank patches from our approach against those from previous work. These and our qualitative results ranging from small image patches to megapixel stylistic images and videos show that our approach better captures the subtle nature in which a style affects content. 1

show abstract

“…Project page: http://www.icst.pku.edu.cn/struct/Projects/UTS.html Style [2]. Non-parametric methods take samples from the style image and place the samples based on pixel intensity [1], [3], [4] or deep features [5] of the target image to synthesize a new image. Parametric methods represent the style as statistical features, and adjust the target image to satisfy these features.…”

Section: Introductionmentioning

confidence: 99%

Context-Aware Text-Based Binary Image Stylization and Synthesis

Yang

2019

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

In this work, we present a new framework for the stylization of text-based binary images. First, our method stylizes the stroke-based geometric shape like text, symbols and icons in the target binary image based on an input style image. Second, the composition of the stylized geometric shape and a background image is explored. To accomplish the task, we propose legibilitypreserving structure and texture transfer algorithms, which progressively narrow the visual differences between the binary image and the style image. The stylization is then followed by a contextaware layout design algorithm, where cues for both seamlessness and aesthetics are employed to determine the optimal layout of the shape in the background. Given the layout, the binary image is seamlessly embedded into the background by texture synthesis under a context-aware boundary constraint. According to the contents of binary images, our method can be applied to many fields. We show that the proposed method is capable of addressing the unsupervised text stylization problem and is superior to stateof-the-art style transfer methods in automatic artistic typography creation. Besides, extensive experiments on various tasks, such as visual-textual presentation synthesis, icon/symbol rendering and structure-guided image inpainting, demonstrate the effectiveness of the proposed method.

show abstract

Split and Match: Example-Based Adaptive Patch Sampling for Unsupervised Style Transfer

Cited by 121 publications

References 24 publications

Neural Style Transfer: A Review

Neural Style Transfer: A Review

A Style-Aware Content Loss for Real-Time HD Style Transfer

Context-Aware Text-Based Binary Image Stylization and Synthesis

Contact Info

Product

Resources

About