Deep Reference Mining From Scholarly Literature in the Arts and Humanities

Abstract

We consider the task of reference mining: the detection, extraction and classification of references within the full text of scholarly publications. Reference mining brings forward specific challenges, such as the need to capture the morphology of highly abbreviated words and the dependence among the elements of a reference, both following codified reference styles. This task is particularly difficult, and little explored, with respect to the literature in the arts and humanities, where references are mostly given in footnotes. We apply a deep learning architecture for reference mining from the full text of scholarly publications. We explore and discuss three architectural components: word and character-level word embeddings, different prediction layers (Softmax and Conditional Random Fields) and multi-task over single-task learning. Our best model uses both pre-trained word embeddings and characters embeddings, and a BiLSTM-CRF architecture. We test our solution on a dataset of annotated references from the historiography on Venice and, using a linear-chain CRF classifier as a baseline, we show that this deep learning architecture improves by a considerable margin. Furthermore, multi-task learning performs almost on par with a single-task approach. We thus confirm that there are important gains to be had by adopting deep learning for the task of reference mining.

From Books to Knowledge Graphs

“…Existing services that implement such end-to-end extraction of references are GROBID [Lopez, 2009], CERMINE [Tkaczyk et al, 2015], BILBO [Kim et al, 2011] and EXCITE [Hosseini et al, 2019], with the last two having a specific focus on AHSS publications. While the large majority of work in this area dealt with references to secondary literature, the extraction of references to primary sources was investigated, in particular with respect to archival references [Rodrigues Alves et al, 2018] and canonical references to Greek and Latin literary works [Romanello, 2015].…”

Named Entity Recognition and Classification on Historical Documents: A Survey

“…Test different static embedding algorithms [178] Test different static embedding granularity [162] Use modern static embeddings (word2vec, fastText)…”

“…[91] Use modern char-level LM embeddings (Flair) [184] Use modern word-level LM embeddings (BERT, ELMo) [70,152,203] Uses stack of modern embeddings [105,137,162] Transfer learning How well modern embeddings can transfer to historical texts? What is the impact of in-domain embeddings?…”

“…Is more task-specific labelled data more helpful than big or in-domain LMs? [162,191] Compare transformers and BiLSTM [25] Table 6. Synthetic view of DL experiments mapped with research questions.…”

“…In the context of reference mining in the arts and humanities, Rodriguez et al [162] also investigate the benefit of BiLSTM over traditional CRFs, and of multiple input representations. Their experiments focus on three architectural components: input layer (word and character-level word embeddings), prediction layers (Softmax and CRF), and learning setting (multi-task and single-task).…”