Estimating the true cost of garden-pathing: A computational model of latent cognitive processes

Abstract

What is the processing cost of being garden-pathed by a temporary syntactic ambiguity? We argue that comparing average reading times in garden-path versus non-garden-path sentences is not enough to answer this question. Trial-level contaminants such as inattention, the fact that garden-pathing may occur non-deterministically in the ambiguous condition, and “triage” (rejecting the sentence without reanalysis; Fodor & Inoue, 2000) lead to systematic underestimates of the true cost of garden-pathing. Furthermore, the pure garden-path effect due to encountering an unexpected word needs to be separated from the additional cost of syntactic reanalysis. To get more realistic estimates for the individual processing costs of garden-path surprisal and syntactic reanalysis, we implement a novel computational model that includes trial-level contaminants as probabilistically occurring latent cognitive processes. The model shows good predictive fit to existing reading-time and judgment data. Furthermore, the latent-process approach captures differences between NP/Z garden-path sentences and RRC garden-path sentences: The NP/Z garden path occurs nearly deterministically but can be mostly eliminated by adding a comma. By contrast, the RRC garden path occurs with a lower probability, but disambiguation via semantic plausibility is not always effective.

Incremental Processing of Principle B: Mismatches Between Neural Models and Humans

“…For example, the GMME reported for humans in the condition without an interaction with Principle B was 63 milliseconds, while GPT-Neo predicted an average of around 5.7 milliseconds and GPT-J an average of around 4.7 milliseconds. Similar results have been obtained in prior work for non-pronominal constructions, suggesting a broader inability for surprisal measures from neural models to capture the processing cost of grammatical violations (van Schijndel and Linzen, 2021;Wilcox et al, 2021b;Paape and Vasishth, 2022).…”

“…Recent work has placed increased attention on finer-grained comparisons between neural models and humans (e.g., van Schijndel and Linzen, 2021;Wilcox et al, 2021b;Paape and Vasishth, 2022). The growing consensus is that neural models underestimate the processing costs seen with humans, while nonetheless capturing the broad patterns (see Wilcox et al, 2021b).…”

The transposed-word effect does not require parallel word processing: Failure to notice transpositions with serial presentation of words

“…While we do not oppose the thesis that language processing is highly incremental, we maintain that the existence of some non-incremental processing has consequences for both linguistic judgments and eye movements (Huang & Staub, 2021a), and might even do so for reading of normal, grammatical sentences. For instance, a recent study by Paape and Vasishth (2022) showed that explicit consideration of qualitatively different types of trials -in contrast to assuming homogeneous incrementality across all trials -provided better predictive fit to reading time and judgment data.…”

Do local coherence effects exist in English reduced relative clauses?

“…Similarly, self-organizing theories predict that local interactions between strings of words can interfere with the formation of the overall structure of the sentence, especially when the local string is semantically coherent (Smith et al, 2021). At present, our results do not adjudicate between these theories, and quantitative model comparisons (see Paape & Vasishth, 2022;Yadav et al, 2023, for examples) will be necessary to determine which theories best explain the overall pattern of local coherence effects.…”