The visual digital turn: Using neural networks to study historical images

Wevers, Melvin; Smits, Thomas

doi:10.1093/llc/fqy085

Cited by 57 publications

(40 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Regarding visual content in historic newspapers in particular, Wevers and Smits utilize Inception v3 embeddings to analyze the CHRONIC and SIAMESET datasets described in Section 2.1. Their work includes deploying SIAMESE, a recommender system for historic newspaper advertisements, and analyzing the training of a new classification layer on top of the Inception embeddings to predict custom categories [66]. Indeed, in addition to supporting visualizations of latent spaces that capture semantic similarity, image embeddings are desirable for visual search and recommendation tasks due to the ability to perform fast similarity querying with them.…”

Section: Image Embeddings and Cultural Heritagementioning

confidence: 99%

The Newspaper Navigator Dataset

Lee

Mears

Jakeway

et al. 2020

Proceedings of the 29th ACM International Conference on Information &Amp; Knowledge Management

View full text Add to dashboard Cite

Chronicling America is a product of the National Digital Newspaper Program, a partnership between the Library of Congress and the National Endowment for the Humanities to digitize historic American newspapers. Over 16 million pages have been digitized to date, complete with high-resolution images and machine-readable METS/ALTO OCR. Of considerable interest to Chronicling America users is a semantified corpus, complete with extracted visual content and headlines. To accomplish this, we introduce a visual content recognition model trained on bounding box annotations collected as part of the Library of Congress's Beyond Words crowdsourcing initiative and augmented with additional annotations including those of headlines and advertisements. We describe our pipeline that utilizes this deep learning model to extract 7 classes of visual content: headlines, photographs, illustrations, maps, comics, editorial cartoons, and advertisements, complete with textual content such as captions derived from the METS/ALTO OCR, as well as image embeddings. We report the results of running the pipeline on 16.3 million pages from the Chronicling America corpus and describe the resulting Newspaper Navigator dataset, the largest dataset of extracted visual content from historic newspapers ever produced. The Newspaper Navigator dataset, finetuned visual content recognition model, and all source code are placed in the public domain for unrestricted re-use.

show abstract

Section: Image Embeddings and Cultural Heritagementioning

confidence: 99%

The Newspaper Navigator Dataset

Lee

Mears

Jakeway

et al. 2020

Proceedings of the 29th ACM International Conference on Information &Amp; Knowledge Management

View full text Add to dashboard Cite

show abstract

“…This complicates our understanding of the digital object from a library perspective, but it also opens up new possibilities for managing those objects and making them discoverable. (Wevers & Smits, 2020;Arnold & Tilton, 2020) As the Collections as Data project attests, the libraries that support computational access to their collections are still few. (Padilla et al, 2019) The shift in thinking of digital materials as data sets rather than as discrete objects is not only about changing how we support users.…”

Section: From Generalizability To Accountabilitymentioning

confidence: 99%

Managing Bias When Library Collections Become Data

Coleman

2020

IJoL

View full text Add to dashboard Cite

Developments in AI research have dramatically changed what we can do with data and how we can learn from data. At the same time, implementations of AI amplify the prejudices in data often framed as ‘data bias’ and ‘algorithmic bias.’ Libraries, tasked with deciding what is worth keeping, are inherently discriminatory and yet remain trusted sources of information. As libraries begin to systematically approach their collections as data, will they be able to adopt and adapt the AI-driven tools to traditional practices? Drawing on the work of the AI initiative within Stanford Libraries, the Fantastic Futures conference on AI for libraries, archives, and museums, and recent scholarship on data bias and algorithmic bias, this article encourages libraries to engage critically with AI and help shape applications of the technology to reflect the ethos of libraries for the benefit of libraries themselves and the patrons they serve. A brief examination of two core concepts in machine learning, generalization and unstructured data, provides points of comparison to library practices in order to uncover the theoretical assumptions driving the different domains. The comparison also offers a point of entry for libraries to adopt machine learning methods on their own terms.

show abstract

“…Drawing on a sample of archived websites between 1999 and 2014, for example, one research study found that in 1999 the percentage of text on an average webpage was only around 22%; peaking in 2005 at 32% and declining to around 25% by 2014 [4]. Elsewhere in the digital humanities, scholars are increasingly attuned to the "visual turn": the need to not only "distantly read" material, but to "distantly view" them as well [1]; examples include applications of neural networks to analyze image collections [13]. Our own previous work has attempted to provide scholars with image access into web archives by taking advantage of object detectors based on neural networks to create collages that portray a multitude of pre-defined objects [14].…”

Section: The Visual Turnmentioning

confidence: 99%

Content-Based Exploration of Archival Images Using Neural Networks

Adewoye

Han

Ruest

et al. 2020

Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020

View full text Add to dashboard Cite

We present DAIRE (Deep Archival Image Retrieval Engine), an image exploration tool based on latent representations derived from neural networks, which allows scholars to "query" using an image of interest to rapidly find related images within a web archive. This work represents one part of our broader effort to move away from text-centric analyses of web archives and scholarly tools that are direct reflections of methods for accessing the live web. This short piece describes the implementation of our system and a case study on a subset of the GeoCities web archive.

show abstract

The visual digital turn: Using neural networks to study historical images

Cited by 57 publications

References 12 publications

The Newspaper Navigator Dataset

The Newspaper Navigator Dataset

Managing Bias When Library Collections Become Data

Content-Based Exploration of Archival Images Using Neural Networks

Contact Info

Product

Resources

About