ArtFID: Quantitative Evaluation of Neural Style Transfer

Wright, Matthias; Ommer, Björn

doi:10.1007/978-3-031-16788-1_34

Cited by 26 publications

(7 citation statements)

References 70 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A vectorized method using CLIP encoder [27] is applied to control the color and the shape properly so that our scheme successfully stylizes the input portrait without identity loss. We apply various quantitative evaluations including FID and ArtFID [28] with the results from existing works and demonstrate the excellence of our scheme.…”

Section: Introductionmentioning

confidence: 94%

See 1 more Smart Citation

Artfusion: A Diffusion Model-Based Style Synthesis Framework for Portraits

Yang,

Min

2024

Electronics

View full text Add to dashboard Cite

We present a diffusion model-based approach that applies the artistic style of an artist or an art movement to a portrait photograph. Learning the style from the artworks of an artist or an art movement requires a training dataset composed of a lot of samples. We resolve this limitation by combining Contrastive Language Image Pretraining (CLIP) encoder and diffusion model, since the CLIP encoder extracts the features from an input portrait in a very effective way. Our framework includes three independent CLIP encoders that extract the text features, color features and Canny edge features from an input portrait, respectively. These features are incorporated to the style information extracted through a diffusion model to complete the stylization on an input portrait. The diffusion model extracts the style information from the sample images in the training dataset using an image encoder. The denoising steps in the diffusion model applies the style information from the training dataset to the CLIP-based features from an input portrait. Finally, our framework produces an artistic portrait that presents both the identity of the input portrait and the artistic style from the training dataset. The most important contribution of our framework is that our framework requires less than a hundred sample images for an artistic style. Therefore, our framework can successfully extract styles from an artist who has drawn less than a hundred artworks. We sample three artists and three art movements and apply these styles to the portraits of various identities and produce visually pleasing results. We evaluate our results using various metrics, including Frechet Inception Distance (FID), ArtFID and Language-Image Quality Evaluator (LIQE) to prove the excellence of our results.

show abstract

Section: Introductionmentioning

confidence: 94%

“…We apply three metrics for the quantitative approach: Frechet Inception Distance (FID), ArtFID [28], and Language-Image Quality Evaluator (LIQE) [37]. FID and ArtFID are used to measure the conceptual distances between two images in many studies.…”

Section: Quantitative Evaluationmentioning

confidence: 99%

Artfusion: A Diffusion Model-Based Style Synthesis Framework for Portraits

Yang,

Min

2024

Electronics

View full text Add to dashboard Cite

show abstract

“…In previous work, focusing on only one of these aspects was common. However, Wright et al 48 proposed a method for quantitatively assessing style transfer models that measures content preservation and style matching and combines them to form a quantitative metric, ArtFID. This approach allows for a more comprehensive assessment of the quality of style transfer.…”

Section: Artfidmentioning

confidence: 99%

BcsUST: universal style transformation network for balanced content styles

Zhao,

Qian,

Song

2023

J. Electron. Imag.

View full text Add to dashboard Cite

Composition, color, and brushstrokes are the primary factors in evaluating and appreciating artwork and are vital points to be considered and addressed in arbitrary style transfer tasks. However, existing methods do not balance these elements well, so we propose an approach that balances content structure and style patterns for universal style transformation (BcsUST). Specifically, two lightweight encoders based on residual networks pass the extracted style and content features into the multi-domain structure attention module, which applies a self-attention mechanism to align the input features point by point followed by manifold alignment. A secondary alignment strategy is used to integrate global styles harmoniously into the semantic structure of the content, while obtaining content details to keep the structure intact. Then, the texture modulator generates style convolution parameters to dynamically adjust the filter within the micro-decoder feature subspace, injecting complex and flexible style signals for stylization to enrich the colors and form exquisite strokes. In addition, this work proposes a two-stage training strategy that introduces an artistic appreciation loss to further balance content structure and stylistic signals. Numerous qualitative and quantitative studies demonstrate that the BcsUST framework produces images that resemble the artists' paintings more closely.

show abstract

“…For the quantitative evaluation of our study, we estimate Frechet Inception Distance (FID) and ArtFID [27] for the TOL images in Figure 9. We estimate both FID and ArtFID for the results from four existing schemes including MUNIT, CycleGAN, ChipGAN and NST are compared with ours for the eight input landscape images.…”

Section: Quantitative Evaluationmentioning

confidence: 99%

TOLGAN: An End-to-end framework for producing Traditional Orient Landscape

Kim,

Yang,

Min

2023

Preprint

View full text Add to dashboard Cite

We present TOLGAN that generates traditional oriental landscape (TOL) image from a map that specifies the locations and shapes of the elements composing TOL. Users can create a TOL map by using a user interface or a segmentation scheme from a photograph. We design the generator of TOLGAN as a series of decoding layers where the map is applied between the layers. The generated TOL image is further enhanced through an AdaIN architecture. The discriminator of TOLGAN processes a generated image and its groundtruth TOL artwork image. TOLGAN is trained through a dataset composed of paired TOL artwork images and their TOL maps. We present a tool through which users can produce a TOL map by specifying and organizing the elements of TOL artworks. TOLGAN successfully generates a series of TOL images from the TOL map. We evaluate our approach using a quantitative way by estimating FID and ArtFID scores and a qualitative way by executing two user studies. Through these studies, we prove the excellence of our approach by comparing our results with those from several important existing works.

show abstract

ArtFID: Quantitative Evaluation of Neural Style Transfer

Cited by 26 publications

References 70 publications

Artfusion: A Diffusion Model-Based Style Synthesis Framework for Portraits

Artfusion: A Diffusion Model-Based Style Synthesis Framework for Portraits

BcsUST: universal style transformation network for balanced content styles

TOLGAN: An End-to-end framework for producing Traditional Orient Landscape

Contact Info

Product

Resources

About