Weakly Supervised Object Detection in Artworks

Gonthier, Nicolas; Gousseau, Yann; Ladjal, Saïd; Bonfait, Olivier

doi:10.1007/978-3-030-11012-3_53

Cited by 59 publications

(50 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the analysis side, there are several efforts on collection and annotation of large-scale art datasets [27,31,34,48,44,30], and using them for genre and authorship classification [27,45,44]. Others focus on applying and generalizing visual correspondence and object detection methods to paintings using both classical [41,18,7,4], as well as deep [6,8,47,20] methods. Most closely related to us is work of Yin et al [49], which used the same Brueghel data [1], annotating it to train detectors for five object categories (carts, cows, windmills, rowboats and sailbaots).…”

Section: Computer Vision and Artmentioning

confidence: 99%

Discovering Visual Patterns in Art Collections With Spatially-Consistent Feature Learning

Shen

Efros

Aubry

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

Our goal in this paper is to discover near duplicate patterns in large collections of artworks. This is harder than standard instance mining due to differences in the artistic media (oil, pastel, drawing, etc), and imperfections inherent in the copying process. Our key technical insight is to adapt a standard deep feature to this task by fine-tuning it on the specific art collection using self-supervised learning. More specifically, spatial consistency between neighbouring feature matches is used as supervisory fine-tuning signal. The adapted feature leads to more accurate style-invariant matching, and can be used with a standard discovery approach, based on geometric verification, to identify duplicate patterns in the dataset. The approach is evaluated on several different datasets and shows surprisingly good qualitative discovery results. For quantitative evaluation of the method, we annotated 273 near duplicate details in a dataset of 1587 artworks attributed to Jan Brueghel and his workshop 1 . Beyond artworks, we also demonstrate improvement on localization on the Oxford5K photo dataset as well as on historical photograph localization on the Large Time Lags Location (LTLL) dataset.

show abstract

Section: Computer Vision and Artmentioning

confidence: 99%

Discovering Visual Patterns in Art Collections With Spatially-Consistent Feature Learning

Shen

Efros

Aubry

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

show abstract

“…Several works have focused on object recognition and detection in artworks [6,7,13,16,36,41]. A first attempt to use deep neural networks for object recognition in visual arts was presented in [12].…”

Section: Related Workmentioning

confidence: 99%

Visual link retrieval and knowledge discovery in painting datasets

Castellano

Lella

Vessio

2020

Multimed Tools Appl

View full text Add to dashboard Cite

Visual arts are of inestimable importance for the cultural, historic and economic growth of our society. One of the building blocks of most analysis in visual arts is to find similarity relationships among paintings of different artists and painting schools. To help art historians better understand visual arts, this paper presents a framework for visual link retrieval and knowledge discovery in digital painting datasets. Visual link retrieval is accomplished by using a deep convolutional neural network to perform feature extraction and a fully unsupervised nearest neighbor mechanism to retrieve links among digitized paintings. Historical knowledge discovery is achieved by performing a graph analysis that makes it possible to study influences among artists. An experimental evaluation on a database collecting paintings by very popular artists shows the effectiveness of the method. The unsupervised strategy makes the method interesting especially in cases where metadata are scarce, unavailable or difficult to collect.

show abstract

“…The application of artificial intelligence techniques to the cultural heritage field has attracted increasing attention in recent years [8,21,22,[28][29][30]39]. Most of these work focus on automatic metadata annotation such as predicting the author, material, and date of an artwork.…”

Section: Introductionmentioning

confidence: 99%

Generating Captions for Images of Ancient Artworks

Sheng

Moens

2019

Proceedings of the 27th ACM International Conference on Multimedia

View full text Add to dashboard Cite

The neural encoder-decoder framework is widely adopted for image captioning of natural images. However, few works have contributed to generating captions for cultural images using this scheme. In this paper, we propose an artwork type enriched image captioning model where the encoder represents an input artwork image as a 512-dimensional vector and the decoder generates a corresponding caption based on the input image vector. The artwork type is first predicted by a convolutional neural network classifier and then merged into the decoder. We investigate multiple approaches to integrate the artwork type into the captioning model among which is one that applies a step-wise weighted sum of the artwork type vector and the hidden representation vector of the decoder. This model outperforms three baseline image captioning models for a Chinese art image captioning dataset on all evaluation metrics. One of the baselines is a state-of-the-art approach fusing textual image attributes into the captioning model for natural images. The proposed model also obtains promising results for another Egyptian art image captioning dataset. CCS CONCEPTS • Information systems → Multimedia content creation; • Computing methodologies → Scene understanding.

show abstract

Weakly Supervised Object Detection in Artworks

Cited by 59 publications

References 46 publications

Discovering Visual Patterns in Art Collections With Spatially-Consistent Feature Learning

Discovering Visual Patterns in Art Collections With Spatially-Consistent Feature Learning

Visual link retrieval and knowledge discovery in painting datasets

Generating Captions for Images of Ancient Artworks

Contact Info

Product

Resources

About