Toward Computational Models of Multilingual Sentence Processing

Abstract

Although computational models can simulate aspects of human sentence processing, research on this topic has remained almost exclusively limited to the single language case. The current review presents an overview of the state of the art in computational cognitive models of sentence processing, and discusses how recent sentence-processing models can be used to study bi-and multilingualism. Recent results from cognitive modeling and computational linguistics suggest that phenomena specific to bilingualism can emerge from systems that have no dedicated components for handling multiple languages. Hence, accounting for human bi-/multilingualism may not require models that are much more sophisticated than those for the monolingual case.

Comparing predictors of sentence self-paced reading times: Syntactic complexity versus transitional probability metrics

“…Our TP measures included bigram and trigram forward and backward TP, obtained from an n-gram model that was trained on unanalyzed word sequences and did not incorporate information about hierarchical sentential syntax ([ 55 ]). In line with previous studies (e.g., [ 4 , 33 , 36 ]), the four TP measures were operationalized as the logarithm of each word’s occurrence probability.…”

“…An important topic in reading research has been the operationalization of sentential complexity. Previous research has led to two main approaches for quantifying complexity: in terms of syntactic complexity (SC), which refers to a set of measures based on hierarchical dependency structures (e.g., [ 1 , 2 ]), and in terms of transitional probability (TP), which refers to a class of information-theoretical metrics concerning probabilistic patterns of co-occurrence of linguistic units (e.g., [ 3 , 4 ]). Crucially, previous empirical reports have provided mixed evidence with regard to the importance of SC and TP in predicting sentence reading speed.…”

Cumulative Frequency Can Explain Cognate Facilitation in Language Models

“…In research on monolingual online sentence processing, language models are commonly used to study a variety of phenomena (e.g., Gulordava, Bojanowski, Grave, Linzen, & Baroni, 2018;Arehalli & Linzen, 2020). In the computational study of bilingualism, to our knowledge, only Frank (2014) and Frank, Trompenaars, and Vasishth (2016) trained such models on natural language data from two languages at the same time (see also Frank, 2021). These two studies found a significant correlation between the bilingual model's predictions and reading times of L1 Dutch and L2 English sentences by Dutch-English bilinguals.…”

A Methodological Review of Machine Learning in Applied Linguistics

“…Monner et al (2013) explained the language learning phenomenon from the perspective of neural networks. Frank (2020) discussed how the recurrent neural networks can enlighten us on multilingual sentence processing models. It can be learned from above that ML algorithms, especially neural networks, have gradually received attention from the researchers specializing in psycholinguistics or neurolinguistics.…”