Lexical prediction in language comprehension: a replication study of grammatical gender effects in Dutch

Abstract

An important question in predictive language processing is the extent to which prediction effects can reliably be measured on pre-nominal material (e.g. articles before nouns). Here, we present a large sample (N = 58) close replication of a study by Otten and van Berkum (2009). They report ERP modulations in relation to the predictability of nouns in sentences, measured on gendermarked Dutch articles. We used nearly identical materials, procedures, and data analysis steps. We fail to replicate the original effect, but do observe a pattern consistent with the original data. Methodological differences between our replication and the original study that could potentially have contributed to the diverging results are discussed. In addition, we discuss the suitability of Dutch gender-marked determiners as a test-case for future studies of pre-activation of lexical items.

Are words pre-activated probabilistically during sentence comprehension? Evidence from new data and a Bayesian random-effects meta-analysis using publicly available data

“…However, as in recent replication attempts (Nieuwland et al, 2018;Ito et al, 2017a;Kochari & Flecken, 2019;Kochari & Flecken, 2019), the patterns we found are in the expected direction in terms of polarity: the more unexpected the gender of the determiner, the larger the negativity of the average amplitude from the typical electrodes and time window of the N400 effect. It is therefore unclear from a qualitative literature review alone how strong the evidence is in favor (or against) pre-activation.…”

“…However, Otten and Van Berkum (2009) detected a negativity for unpredictable determiners in comparison with predictable determiners with visual presentation, similarly to what was detected in Otten et al (2007) (with adjectives and auditory presentation). Kochari and Flecken (2019) attempted a replication of Otten and Van Berkum (2009) with a larger sample size ( = 70 vs. = 38), and while they failed to find a significant effect at the gender-marked determiner, they did observe a pattern consistent with the original experiment. Recently, Fleur, Flecken, Rommers, and Nieuwland (2019) manipulated gender expectancy and whether the expected noun phrase would be definite or not in Dutch, and detected an N400-like effect for the gender manipulation.…”

“…Thus, from our experiment alone, we should conclude that there is no evidence for the pre-activation of features of expected nouns, measured on gender-marked articles in German. Given that results from recent studies are either inconclusive (Kochari & Flecken, 2019;Ito et al, 2017a), or weakly supporting the null (Nieuwland et al, 2018), if there is such pre-activation, its effect is too small to be detected reliably by a single experiment, even if the sample size is very large by EEG standards (e.g., 334 participants in Nieuwland et al, 2018). Furthermore, a self-paced reading study with a relatively large-sample size (120 participants and 60 items) has shown that unexpected gender-marking in determiners in Spanish do not lead to a slowdown in comparison to expected ones, and thus if pre-activation of gender marking occurs it may have no measurable impact in reading times (Guerra, Nicenboim, & Helo, 2018).…”

“…For the meta-analysis, we included the publicly available data from Nieuwland et al (2018), Ito et al (2017a), and Kochari and Flecken (2019), and a subset of our own data (only data from items with constraining contexts). We reanalyzed the publicly available data using Bayesian linear mixed models that were as similar as possible: For all the studies, the predictor labeled predictability is smoothed, transformed to log base two, and then scaled using the same mean and standard deviation as in our data.…”

Implicit semantic prediction error can account for N400 effects on articles that do not differ in meaning: A neural network model

“…First, the simulations provide a mechanistically explicit account of article induced N400 effects that does not depend on prediction at the level of word forms, but rather treats N400 amplitudes as the change in the conditional probabilities of semantic features cued by encountering the articles (Rabovsky et al, 2018;Rabovsky & McRae, 2014;Yan, Kuperberg, & Jaeger, 2017). This explanation differs from the most common interpretation of these effects Hagoort, 2017;Lau et al, 2008;Pickering & Garrod, 2013) and can, from the perspective implemented in the model, also be applied to related findings concerning article gender (see Kochari &Nicenboim et al, 2019 for recent reviews; see Fleur, Flecken, Rommers, & Nieuwland, 2019 for evidence consistent with the view that word form prediction may not be essential).…”