Categorizing paintings in art styles based on qualitative color descriptors, quantitative global features and machine learning (QArt-Learn)

Falomir, Zoe; Museros, Lledó; Sanz, Ismael; Abril, Luis González

doi:10.1016/j.eswa.2017.11.056

Cited by 42 publications

(19 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The task of classifying paintings by artist has later been addressed in different studies (Cetinic & Grgic, 2013), as well as the challenge of visualizing similarities (Bressan et al, 2008;Shamir et al, 2010;Shamir & Tarakhovsky, 2012) and exploring influential connections among artists . Most of the earlier studies that addressed the topic of artist and other art-related tasks such as style (Lombardi, 2005;Arora & Elgammal, 2012;Falomir et al, 2018) and genre classification (Zujovic et al, 2009), share one similar methodology. Their approach usually includes extracting a set of various image features and using them to train different classifiers such as support vector machines (SVM), multilayer perceptron (MLP) or k-nearest neighbours (k-NN).…”

Section: Related Workmentioning

confidence: 99%

Fine-tuning Convolutional Neural Networks for fine art classification

Cetinić

Lipić

Grgić

2018

Expert Systems with Applications

190

View full text Add to dashboard Cite

The increasing availability of large digitized fine art collections opens new research perspectives in the intersection of artificial intelligence and art history. Motivated by the successful performance of Convolutional Neural Networks (CNN) for a wide variety of computer vision tasks, in this paper we explore their applicability for art-related image classification tasks. We perform extensive CNN fine-tuning experiments and consolidate in one place the results for five different art-related classification tasks on three large fine art datasets. Along with addressing the previously explored tasks of artist, genre, style and time period classification, we introduce a novel task of classifying artworks based on their association with a specific national artistic context. We present state-of-the-art classification results of the addressed tasks, signifying the impact of our method on computational analysis of art, as well as other image classification related research areas. Furthermore, in order to question transferability of deep representations across various source and target domains, we systematically compare the effects of domain-specific weight initialization by evaluating networks pre-trained for different tasks, varying from object and scene recognition to sentiment and memorability labelling. We show that fine-tuning networks pre-trained for scene recognition and sentiment prediction yields better results than fine-tuning networks pre-trained for object recognition. This novel outcome of our work suggests that the semantic correlation between different domains could be inherent in the CNN weights. Additionally, we address the practical applicability of our results by analysing different aspects of image similarity. We show that features derived from fine-tuned networks can be employed to retrieve images similar in either style or content, which can be used to enhance capabilities of search systems in different online art collections.

show abstract

Section: Related Workmentioning

confidence: 99%

Fine-tuning Convolutional Neural Networks for fine art classification

Cetinić

Lipić

Grgić

2018

Expert Systems with Applications

190

View full text Add to dashboard Cite

show abstract

“…For classification tasks, the traditional approach is to first extract features identified by art and computer experts as being the most representative and then to use machine learning techniques for image categorization. For example, to categorize painting styles, the QArt-learn approach [ 3 ] was established based on qualitative color descriptors (QCD), color similarity (SimQCD), and quantitative global features. K-nearest neighbor (KNN) and support vector machine (SVM) models have been used to classify paintings in the Baroque, Impressionism and Post-Impressionism styles.…”

Section: Introductionmentioning

confidence: 99%

Compare the performance of the models in art classification

et al. 2021

View full text Add to dashboard Cite

Because large numbers of artworks are preserved in museums and galleries, much work must be done to classify these works into genres, styles and artists. Recent technological advancements have enabled an increasing number of artworks to be digitized. Thus, it is necessary to teach computers to analyze (e.g., classify and annotate) art to assist people in performing such tasks. In this study, we tested 7 different models on 3 different datasets under the same experimental setup to compare their art classification performances when either using or not using transfer learning. The models were compared based on their abilities for classifying genres, styles and artists. Comparing the result with previous work shows that the model performance can be effectively improved by optimizing the model structure, and our results achieve state-of-the-art performance in all classification tasks with three datasets. In addition, we visualized the process of style and genre classification to help us understand the difficulties that computers have when tasked with classifying art. Finally, we used the trained models described above to perform similarity searches and obtained performance improvements.

show abstract

“…Different combinations of features and classifiers were explored in [7]. In a recent work [8], the classification of three artistic styles based on the qualitative color descriptors and color similarity was explored using the SVM and the k-Nearest Neighbors (k-NN) as classifiers. In [9] unsupervised feature extraction was investigated, for the classification of 6776 paintings into eight stylistic groups.…”

Section: Related Workmentioning

confidence: 99%

Two-Stage Deep Learning Approach to the Classification of Fine-Art Paintings

2019

View full text Add to dashboard Cite

Due to the digitization of fine art collections, pictures of fine art objects stored at museums and art galleries became widely available to the public. It created a demand for efficient software tools that would allow rapid retrieval and semantic categorization of art. This paper introduces a new, two-stage image classification approach aiming to improve the style classification accuracy. At the first stage, the proposed approach divides the input image into five patches and applies a deep convolutional neural network (CNN) to train and classify each patch individually. At the second stage, the outcomes from the individual five patches are fused in the decision-making module, which applies a shallow neural network trained on the probability vectors given by the first-stage classifier. While the first stage categorizes the input image based on the individual patches, the second stage infers the final decision label categorizing the artistic style of the analyzed input image. The key factor in improving the accuracy compared to the baseline techniques is the fact that the second stage is trained independently on the first stage using probability vectors instead of images. This way, the second stage is effectively trained to compensate for the potential mistakes made during the first stage. The proposed method was tested using six different pre-trained CNNs (AlexNet, VGG-16, VGG-19, GoogLeNet, ResNet-50, and Inceptionv3) as the first-stage classifiers, and a shallow neural network as a second-stage classifier. The experiments conducted using three standard art classification datasets indicated that the proposed method presents a significant improvement over the existing baseline techniques. INDEX TERMS Fine art style recognition, painting classification, machine learning, multi-stage classification, transfer learning, digital humanities.

show abstract

Categorizing paintings in art styles based on qualitative color descriptors, quantitative global features and machine learning (QArt-Learn)

Cited by 42 publications

References 22 publications

Fine-tuning Convolutional Neural Networks for fine art classification

Fine-tuning Convolutional Neural Networks for fine art classification

Compare the performance of the models in art classification

Two-Stage Deep Learning Approach to the Classification of Fine-Art Paintings

Contact Info

Product

Resources

About