Restoration of fragmentary Babylonian texts using recurrent neural networks

Fetaya, Ethan; Lifshitz, Yonatan; Aaron, Elad; Gordin, Shai

doi:10.1073/pnas.2003794117

Cited by 31 publications

(26 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…We use two strong baselines: (1) the LSTM model that was proposed by Fetaya et al (2020), and was retrained on our dataset using their default configuration; 4,5 and (2) the cased BERT-base multilingual model, without finetuning over Oracc. 6 We compare these two baselines against our models, as presented in 4.2, trained in three configurations: (1) BERT+AKK(mono) refers to the reduced size BERT model, trained from scratch on the Akkadian texts from Oracc; (2) MBERT+Akk is a finetuned version of M-BERT on the Akkadian texts, using the model's additional free tokens to encode sub-word tokens from Oracc; and…”

Section: Models and Datasetsmentioning

confidence: 99%

See 1 more Smart Citation

Filling the Gaps in Ancient Akkadian Texts: A Masked Language Modelling Approach

Lazar¹,

Saret²,

Yehudai³

et al. 2021

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

We present models which complete missing text given transliterations of ancient Mesopotamian documents, originally written on cuneiform clay tablets (2500 BCE -100 CE). Due to the tablets' deterioration, scholars often rely on contextual cues to manually fill in missing parts in the text in a subjective and time-consuming process. We identify that this challenge can be formulated as a masked language modelling task, used mostly as a pretraining objective for contextualized language models. Following, we develop several architectures focusing on the Akkadian language, the lingua franca of the time. We find that despite data scarcity (1M tokens) we can achieve state of the art performance on missing tokens prediction (89% hit@5) using a greedy decoding scheme and pretraining on data from other languages and different time periods. Finally, we conduct human evaluations showing the applicability of our models in assisting experts to transcribe texts in extinct languages.

show abstract

Section: Models and Datasetsmentioning

confidence: 99%

“…Most related to our work, Fetaya et al (2020) designed an LSTM model which similarly aims to complete fragmentary sequences in Babylonian texts. They differ from us in two major aspects.…”

Section: Related Workmentioning

confidence: 99%

Filling the Gaps in Ancient Akkadian Texts: A Masked Language Modelling Approach

Lazar¹,

Saret²,

Yehudai³

et al. 2021

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

show abstract

“…Besides emotion and happiness, dictionary-driven representations are also extensively used to detect depression in social media [105,158] . Despite the wide adoption of dictionary-driven representations, Jaidka et al [77] made a comparison between unsupervised dictionary-driven and supervised data-driven methods, and verified that the latter is more robust for well-being estimation from social media data. Therefore, outside of the dictionary-driven representations, linguistic features, such as n-gram and BOW, are also applied to represent text in psychology, combined with the supervised machine learning method.…”

Section: A11 Symbol-based Representationmentioning

confidence: 99%

“…Topic model based representation is often operated as one of the features for the classification task, since it can supply the semantic information of text. For instance, Jaidka et al [77] leveraged the representation learned from LDA model to predict the subjective well-being from Twitter. Besides, Eichstaedt et al [51] also used LDA to represent posts on Facebook and predict the depression of users.…”

mentioning

confidence: 99%

From Symbols to Embeddings: A Tale of Two Representations in Computational Social Science

et al. 2021

View full text Add to dashboard Cite

Computational Social Science (CSS), aiming at utilizing computational methods to address social science problems, is a recent emerging and fast-developing field. The study of CSS is data-driven and significantly benefits from the availability of online user-generated contents and social networks, which contain rich text and network data for investigation. However, these large-scale and multi-modal data also present researchers with a great challenge: how to represent data effectively to mine the meanings we want in CSS?To explore the answer, we give a thorough review of data representations in CSS for both text and network.Specifically, we summarize existing representations into two schemes, namely symbol-based and embeddingbased representations, and introduce a series of typical methods for each scheme. Afterwards, we present the applications of the above representations based on the investigation of more than 400 research articles from 6 top venues involved with CSS. From the statistics of these applications, we unearth the strength of each kind of representations and discover the tendency that embedding-based representations are emerging and obtaining increasing attention over the last decade. Finally, we discuss several key challenges and open issues for future directions. This survey aims to provide a deeper understanding and more advisable applications of data representations for CSS researchers.

show abstract

“…The aforementioned process also highlights other significant start-up costs that come with AIG: first, in the form described above, AIG is anything but automatic and extensive preparatory steps are required before any automation takes place (Kosh et al, 2019). Second, in order to actually use computer algorithms, one must either be able to train them, which requires expertise and large computational storing and processing capacities (Fetaya et al, 2020)-or gain access to existing software-which has hitherto not been widely available (Royal et al, 2018).…”

Section: Automated Item Generation (Aig)mentioning

confidence: 99%

Let the Algorithm Speak: How to Use Neural Networks for Automatic Item Generation in Psychological Scale Development

Götz¹,

Maertens²,

Linden³

2021

Preprint

View full text Add to dashboard Cite

Measurement is at the heart of scientific research. As many—perhaps most—psychological constructs cannot be directly observed, there is a steady demand for sound self-report scales to assess such latent constructs. However, scale development is a tedious process that requires researchers to produce good items in large quantities. In the current tutorial, we introduce, explain, and apply the Psychometric Item Generator (PIG), an open-source, free-to-use, self-sufficient natural language processing algorithm that produces large-scale, human-like, customised text output within a few mouse clicks. The PIG is based on the GPT-2, a powerful generative language model, and runs on Google Colaboratory—an interactive virtual notebook environment that executes code on state-of-the-art virtual machines at no cost. We demonstrate that based on an input of three sentences, the PIG produces 65 items that pass initial face validity checks within a single iteration of code and a runtime of less than one minute. The PIG does not require any prior coding skills or access to computational resources and can be easily tailored to any desired context by simply switching out short linguistic prompts in a single line of code. Additionally, the PIG can also be used as a bottom-up tool to expand and diversify the conceptual understanding of a construct or derive hypotheses about its relationships to other, existing constructs. In short, we present an effective, novel machine learning solution to an old psychological challenge. As such, the PIG will not only not require you to learn a new language—but speak yours.

show abstract

Restoration of fragmentary Babylonian texts using recurrent neural networks

Cited by 31 publications

References 27 publications

Filling the Gaps in Ancient Akkadian Texts: A Masked Language Modelling Approach

Filling the Gaps in Ancient Akkadian Texts: A Masked Language Modelling Approach

From Symbols to Embeddings: A Tale of Two Representations in Computational Social Science

Let the Algorithm Speak: How to Use Neural Networks for Automatic Item Generation in Psychological Scale Development

Contact Info

Product

Resources

About