Unveiling the secrets of paintings: deep neural networks trained on high-resolution multispectral images for accurate attribution and authentication

Abstract

Attribution and authentication of paintings are difficult tasks, often based on human expertise. In this work, we present SpectrumArt: a new dataset of multispectral (13 channels) image patches of paintings acquired at very high resolution (800 pixels per mm 2 ). We train deep neural networks on SpectrumArt for attribution (i.e., authorship classification) and authentication (i.e., whether of undisputed origin). For attribution, we obtain an accuracy of 92% on a test set of patches coming from unseen paintings. We also propose two classification metrics for attribution of full paintings based on the prediction for the patches: majority vote and entropy weighted vote. Both metrics lead to an attribution score of 100% on unseen paintings. For authenticity testing, our model agrees with the experts' conclusions on genuine and fake paintings, and provides new insights into the authenticity of paintings where the expert community is divided by proposing a spectral matching score between the painting and an artist. To validate the important advantage of our data collection method, we show that the use of 13 channels instead of 3 and the high resolution of the data significantly improve the accuracy of our models.