Bayesian Hierarchical Finite Mixture Models of Reading Times: A Case Study

Abstract

This theoretical note presents a case study demonstrating the importance of Bayesian hierarchical mixture models as a modelling tool for evaluating the predictions of competing theories of cognitive processes. This note also contributes to improving current practices in data analysis in the psychological sciences. As a case study, we revisit two published data sets from psycholinguistics. In sentence comprehension, it is widely assumed that the distance between linguistic co-dependents affects the latency of dependency resolution: the longer the distance, the longer the time taken to complete the dependency (e.g., Gibson 2000). An alternative theory, direct access (McElree, 2000), assumes that retrieval times are a mixture of two distributions : one distribution represents successful retrievals and the other represents an initial failure to retrieve the correct dependent, followed by a reanalysis (McElree, 1993) that leads to successful retrieval. Here, dependency distance has the effect that in long-distance conditions the proportion of reanalyses is higher (due to similarity-based interference). We implement both theories as Bayesian hierarchical models and show that the direct-access model fits the Chinese relative clause reading time data better than the dependency-distance account. This work makes several novel contributions. First, we demonstrate how the researcher can reason about the underlying generative process of their data, thereby expressing the underlying cognitive process as a statistical model. Second, we show how models that have been developed in an exploratory manner to represent different underlying generative processes can be compared in terms of their predictive performance, using both K-fold cross validation on existing data, and using completely new data. Finally, we show how the models can be evaluated using simulated data; this is a method that is standardly used in Bayesian statistics, but remains unutilized in data analysis within the psychological sciences.

Bayesian data analysis in the phonetic sciences: A tutorial introduction

“…For example, if there are some (say, 5%) 0 ms VOTs in a data-set (e.g., due to speech errors or some other reason), or if there is a mixture of distributions generating the data (as in the case of the English voiced stops produced by the 13 speakers who had prevoicing in some of their tokens), and the model assumes a Gaussian likelihood, the posterior predictive distributions and the distribution of the data will not line up. For a real-life example of such a situation, see Vasishth et al (2017). In Figure 7, we use the Mandarin data to simulate such a situation by randomly replacing 5% of the data with 0 ms values.…”

“…This method of model selection is useful, however, when one is interested in comparing the predictive performance of very different competing models. For fully worked examples of this approach (with reproducible code and data) in the context of cognitive modeling in psycholinguistics, see Nicenboim and Vasishth (2018) and Vasishth et al (2017).…”

Exploratory and Confirmatory Analyses in Sentence Processing: A Case Study of Number Interference in German

“…Second, Bayesian procedures allow us to fit virtually any kind of distribution in a straightforward way. In the past, we have fit hierarchical mixture models (Nicenboim & Vasishth, 2018;Vasishth, Nicenboim, Chopin, & Ryder, 2017) and hierarchical measurement error models (Nicenboim, Roettger, & Vasishth, 2017;Vasishth, Beckman, Nicenboim, Li, & Kong, 2017). In this paper, we fit shifted lognormal mixed models, which lie outside the class of generalized linear models.…”

Exploratory and confirmatory analyses in sentence processing: A case study of number interference in German

“…Second, Bayesian procedures allow us to fit virtually any kind of distribution in a straightforward way. In the past, we have fit hierarchical mixture models Vasishth, Nicenboim, Chopin, & Ryder, 2017), and hierarchical measurement error models (Nicenboim, Roettger, & Vasishth, 2017;Vasishth, Beckman, Nicenboim, Li, & Kong, 2017). In this paper, we fit shifted lognormal mixed models, which lie outside the class of generalized linear models.…”