Why the A/AN prediction effect may be hard to replicate: A rebuttal to DeLong, Urbach & Kutas (2017)

Abstract

In our recent publication “How robust are prediction effects in language comprehension? Failure to replicate article-elicited N400 effects” (Ito, Martin, & Nieuwland, 2016a), we report two experiments which failed to replicate existing ERP evidence for prediction as reported in C. D. Martin et al. (2013), whose study resembled DeLong, Urbach, and Kutas (2005; from hereon DUK05). DeLong, Urbach, and Kutas (2017; from hereon, DUK17) recently published a commentary which depicts our publication as a case of poor scholarship that makes “no substantive contribution to the literature on what factors may matter for prediction and when.” DUK17 warn that the readers of our work “will be led into serious error.” In this rebuttal, we first present evidence that is inconsistent with the arguments of DUK17 regarding our own experiments, and then we briefly discuss other indications why it might be hard to observe and thus replicate the a/an prediction effect.

Prediction of phonological and gender information: An event-related potential study in Italian

“…An exploratory Bayes factor analysis of the N400-cloze correlation suggested that any effect size for phonological prediction would be quite small. These studies raise the question of how robust the a/an article effects are (for more discussions about the replicability of these effects, see DeLong, Urbach, & Kutas, 2017;Ito, Martin, & Nieuwland, 2017b;Nieuwland et al, 2018;Yan, Kuperberg, & Jaeger, 2017). As with gender prediction, there has been inconsistency in the ERPs reported for phonological mismatch.…”

“…Another possible reason for the lack of phonological-mismatch effects relates specifically to the a/an manipulation. The a/an phonological rule is not realised by agreement between the article and the noun but between the article and the initial phoneme of the next word, and the occurrence of an intervening word is very common in English (67% written, 30% spoken, Ito et al, 2017b;Yan et al, 2017). If people make phonological predictions only for the next upcoming word, then the probability that the expected noun comes directly after the article should play an important role in determining when or whether such phonological information should be pre-activated.…”

“…However, in Italian, the likelihood of an article to be immediately followed by a noun is high (70-87%). Estimates from English language corpora suggest that a/an articles are immediately followed by a noun only 33% of the time both in written American and British English (Ito et al, 2017b), though this probability estimate goes up to 70% in speech (Yan et al, 2017). While we do not interpret our results as suggesting that people are more likely to predict specific words in Italian (e.g., than in English) due to the article properties in Italian, articles in Italian are probably more reliable cues about the first phoneme(s) of the upcoming noun than articles in English, and it is possible that the late negativity we found indicates disconfirmation of the predicted noun.…”

The involvement of tonal information during speech prediction: evidence based on Chinese tone sandhi

“…They observed a rather robust N400 effect after the onset of the noun (e.g., 'kite'), but only one lab replicated the N400 effect after the onset of the article (e.g., 'a'). This finding suggests that the weak prediction effect of word-form information (e.g., phonology) might only occur occasionally and such an effect might require a large sample size to be detected (Ito et al, 2017b;Nieuwland et al, 2018). Moreover, some bilingual studies have also observed an unstable pattern of phonological prediction effect for native speakers (L1).…”

The functional significance of the P600

“…Incorporating these findings, there is now no LAN, the N400 is elicited early, by both syntactic and semantic conditions, and the P600 is elicited late, by both syntactic and semantic conditions as well. Nevertheless, the functional significance of both the N400 and the P600, in terms of the underlying cognitive processes, is still a topic of hot debate (e.g., Shen et al [ 6 ] and Ito et al [ 7 ] ). In this context, [ 6 ] have shown, that a semantic P600 paradigm source localized to executive rather than classical language areas, thus supporting an extra-linguistic rather than an intra-linguistic interpretation of the P600.…”