A Neural Algorithm of Artistic Style

Abstract

In fine art, especially painting, humans have mastered the skill to create unique visual experiences through composing a complex interplay between the content and style of an image. Thus far the algorithmic basis of this process is unknown and there exists no artificial system with similar capabilities. However, in other key areas of visual perception such as object and face recognition near-human performance was recently demonstrated by a class of biologically inspired vision models called Deep Neural Networks. 1, 2 Here we introduce an artificial system based on a Deep Neural Network that creates artistic images of high perceptual quality. The system uses neural representations to separate and recombine content and style of arbitrary images, providing a neural algorithm for the creation of artistic images. Moreover, in light of the striking similarities between performance-optimised artificial neural networks and biological vision, 3-7 our work offers a path forward to an algorithmic understanding of how humans create and perceive artistic imagery.

Fast Video Artistic Transfer via Motion Compensation

“…The algorithm eliminates the unnecessary processing of redundant information in consecutive frames, what reduces the overall processing time. Visual results are good and comparable to previous works [2,3,4].…”

“…The temporal segmentation and motion estimation steps of the proposed method were implemented in Matlab R2015a. For comparison, we used Python implementations of the style transfer methods provided by their authors in [2,3,4]. To test the proposed algorithm, we used three videos of the LIVE Quality Video Database [5].…”

“…For example, the image in the 3rd column of the 2nd row corresponds to the stylized version of a frame of video 2, using the artistic style 1. Figure 3 shows results of applying the algorithms proposed by Gatys [2], Johnson [3], and Ruder [4], alongside with the corresponding proposed method adaptations, on 'Video 1' (non-distorted version of MC sequence in LIVE database). In this figure, the results correspond to the 245-th frame of 'Video 1', classified as an inter-frame.…”

“…Gaty's [2] or Johnson's [3]) is applied to convert the original natural intra-frames into an artistic style frames. More specifically, for each x a t reference frame, we compute the corresponding artistic style frame y a t .…”

“…More recently, machine learning algorithms have been used to perform the conversion of natural images into artistic images. For example, Gatys et al [2] developed an algorithm based on neural networks, which captures the style of paintings (or art works) and transfers it to natural images. Although effective, their approach uses a deep convolutional network to mimic the artistic style and transfer it to the natural content, what is computationally expensive.…”