A CNN-LSTM framework for authorship classification of paintings

Jangtjik, Kevin Alfianto; Ho, Trang-Thi; Yeh, Mei-Chen; Hua, Kai-Lung

doi:10.1109/icip.2017.8296806

Cited by 11 publications

(6 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Artist recognition based images of paper prints of artworks were performed using the average score obtained by independent CNNs trained on different image scales. A similar artist recognition approach was reported in [24] where a multiscale pyramid framework comprised of three layers was applied. Fixed-size input images were analyzed by the first CNN layer, while the second layer analyzed four patches, and the third layer analyzed sixteen patches extracted from the input images enlarged by two and four times, respectively.…”

Section: Related Workmentioning

confidence: 99%

Two-Stage Deep Learning Approach to the Classification of Fine-Art Paintings

2019

View full text Add to dashboard Cite

Due to the digitization of fine art collections, pictures of fine art objects stored at museums and art galleries became widely available to the public. It created a demand for efficient software tools that would allow rapid retrieval and semantic categorization of art. This paper introduces a new, two-stage image classification approach aiming to improve the style classification accuracy. At the first stage, the proposed approach divides the input image into five patches and applies a deep convolutional neural network (CNN) to train and classify each patch individually. At the second stage, the outcomes from the individual five patches are fused in the decision-making module, which applies a shallow neural network trained on the probability vectors given by the first-stage classifier. While the first stage categorizes the input image based on the individual patches, the second stage infers the final decision label categorizing the artistic style of the analyzed input image. The key factor in improving the accuracy compared to the baseline techniques is the fact that the second stage is trained independently on the first stage using probability vectors instead of images. This way, the second stage is effectively trained to compensate for the potential mistakes made during the first stage. The proposed method was tested using six different pre-trained CNNs (AlexNet, VGG-16, VGG-19, GoogLeNet, ResNet-50, and Inceptionv3) as the first-stage classifiers, and a shallow neural network as a second-stage classifier. The experiments conducted using three standard art classification datasets indicated that the proposed method presents a significant improvement over the existing baseline techniques. INDEX TERMS Fine art style recognition, painting classification, machine learning, multi-stage classification, transfer learning, digital humanities.

show abstract

Section: Related Workmentioning

confidence: 99%

Two-Stage Deep Learning Approach to the Classification of Fine-Art Paintings

2019

View full text Add to dashboard Cite

show abstract

“…In the last century, there have been a growing number of scientific techniques for attribution and authentication of paintings, including X-ray fluorescence, 1 optical microscopy, 2 Fourier Transformed Infrared Spectrophotometry, 3 Statistical Modeling, 4 and more recently Deep Learning, [5][6][7] which has become the tool of choice for a lot of computer vision tasks. 8 Another line of work is the study of paintings using hyperspectral imaging, [9][10][11] where data are acquired over more than the classical RGB channels.…”

Section: Introductionmentioning

confidence: 99%

Unveiling the secrets of paintings: deep neural networks trained on high-resolution multispectral images for accurate attribution and authentication

E.¹,

Sander

Maxime³

2023

Sixteenth International Conference on Quality Control by Artificial Vision

View full text Add to dashboard Cite

Attribution and authentication of paintings are difficult tasks, often based on human expertise. In this work, we present SpectrumArt: a new dataset of multispectral (13 channels) image patches of paintings acquired at very high resolution (800 pixels per mm 2 ). We train deep neural networks on SpectrumArt for attribution (i.e., authorship classification) and authentication (i.e., whether of undisputed origin). For attribution, we obtain an accuracy of 92% on a test set of patches coming from unseen paintings. We also propose two classification metrics for attribution of full paintings based on the prediction for the patches: majority vote and entropy weighted vote. Both metrics lead to an attribution score of 100% on unseen paintings. For authenticity testing, our model agrees with the experts' conclusions on genuine and fake paintings, and provides new insights into the authenticity of paintings where the expert community is divided by proposing a spectral matching score between the painting and an artist. To validate the important advantage of our data collection method, we show that the use of 13 channels instead of 3 and the high resolution of the data significantly improve the accuracy of our models.

show abstract

“…To avoid high-risk stocks and secure higher profits, a stock trader has only one means of evaluating a company’s performance before purchasing its stock. With the development of technological advances, deep learning—especially convolutional neural networks (CNNs)—has exhibited favorable performance in a range of research fields [ 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 ]. Many researchers have applied deep learning to the question of stock market prediction.…”

Section: Introductionmentioning

confidence: 99%

Stock Price Movement Prediction Using Sentiment Analysis and CandleStick Chart Representation

Huang

2021

Sensors

Self Cite

View full text Add to dashboard Cite

Determining the price movement of stocks is a challenging problem to solve because of factors such as industry performance, economic variables, investor sentiment, company news, company performance, and social media sentiment. People can predict the price movement of stocks by applying machine learning algorithms on information contained in historical data, stock candlestick-chart data, and social-media data. However, it is hard to predict stock movement based on a single classifier. In this study, we proposed a multichannel collaborative network by incorporating candlestick-chart and social-media data for stock trend predictions. We first extracted the social media sentiment features using the Natural Language Toolkit and sentiment analysis data from Twitter. We then transformed the stock’s historical time series data into a candlestick chart to elucidate patterns in the stock’s movement. Finally, we integrated the stock’s sentiment features and its candlestick chart to predict the stock price movement over 4-, 6-, 8-, and 10-day time periods. Our collaborative network consisted of two branches: the first branch contained a one-dimensional convolutional neural network (CNN) performing sentiment classification. The second branch included a two-dimensional (2D) CNN performing image classifications based on 2D candlestick chart data. We evaluated our model for five high-demand stocks (Apple, Tesla, IBM, Amazon, and Google) and determined that our collaborative network achieved promising results and compared favorably against single-network models using either sentiment data or candlestick charts alone. The proposed method obtained the most favorable performance with 75.38% accuracy for Apple stock. We also found that the stock price prediction achieved more favorable performance over longer periods of time compared with shorter periods of time.

show abstract

A CNN-LSTM framework for authorship classification of paintings

Cited by 11 publications

References 8 publications

Two-Stage Deep Learning Approach to the Classification of Fine-Art Paintings

Two-Stage Deep Learning Approach to the Classification of Fine-Art Paintings

Unveiling the secrets of paintings: deep neural networks trained on high-resolution multispectral images for accurate attribution and authentication

Stock Price Movement Prediction Using Sentiment Analysis and CandleStick Chart Representation

Contact Info

Product

Resources

About