Factorial comparison of working memory models.

Abstract

Three questions have been prominent in the study of visual working memory limitations: (a) What is the nature of mnemonic precision (e.g., quantized or continuous)? (b) How many items are remembered? (c) To what extent do spatial binding errors account for working memory failures? Modeling studies have typically focused on comparing possible answers to a single one of these questions, even though the result of such a comparison might depend on the assumed answers to both others. Here, we consider every possible combination of previously proposed answers to the individual questions. Each model is then a point in a 3-factor model space containing a total of 32 models, of which only 6 have been tested previously. We compare all models on data from 10 delayed-estimation experiments from 6 laboratories (for a total of 164 subjects and 131,452 trials). Consistently across experiments, we find that (a) mnemonic precision is not quantized but continuous and not equal but variable across items and trials; (b) the number of remembered items is likely to be variable across trials, with a mean of 6.4 in the best model (median across subjects); (c) spatial binding errors occur but explain only a small fraction of responses (16.5% at set size 8 in the best model). We find strong evidence against all 6 documented models. Our results demonstrate the value of factorial model comparison in working memory.

Changing concepts of working memory

“…Error distributions in delayed estimation are predicted considerably better by a variable-precision model than by alternative models, including the discrete-representation model 15,16,70 . In particular, the model accounts for the increase in the ‘guessing rate’ with increasing set size: when a normal + uniform mixture is fitted to recall errors, low-precision trials will be absorbed into the uniform component, even though they might not represent true guesses.…”

“…However, interpretation of the parameters of the normal + uniform mixture critically depends on the validity of the mixture fit. Careful comparison with data suggests the normal + uniform mixture provides a relatively poor fit to experimental error distributions 15,16,70 and SD normal may therefore systematically underestimate the true variability in memory (Fig. 5).…”

“…However, such models should ideally be informed by biological plausibility and neural data. A recent paper, comparing 32 models in a three-factor space, found that the most successful models incorporated both continuous, variable precision and spatial binding errors, with additional evidence for variability in the number of items stored 70 .…”

“Plateau”-related summary statistics are uninformative for comparing working memory models

“…In recent years, many papers have debated the nature of working memory limitations (including Alvarez & Cavanagh, 2004; Anderson & Awh, 2012; Anderson, Vogel, & Awh, 2011; Bays, Catalao, & Husain, 2009; Bays, Gorgoraptis, Wee, Marshall, & Husain, 2011; Bays & Husain, 2008; Buschman, Siegel, Roy, & Miller, 2011; Donkin, Nosofsky, Gold, & Shiffrin, 2013; Elmore et al, 2011; Fougnie, Suchow, & Alvarez, 2012; Fukuda, Awh, & Vogel, 2010; Heyselaar, Johnston, & Pare, 2011; Keshvari, Van den Berg, & Ma, 2013; Lara & Wallis, 2012; Luck & Vogel, 2013; Rouder, Morey, Cowan, Morey, & Pratte, 2008; Sims, Jacobs, & Knill, 2012; Van den Berg, Awh, & Ma, 2014; Van den Berg, Shin, Chou, George, & Ma, 2012; Wilken & Ma, 2004; Zhang & Luck, 2008). A key aspect of this debate has been whether or not there exists an upper limit to the number of items that can be held in visual working memory.…”

Quality and accessibility of visual working memory during cognitive control of attentional guidance: A Bayesian model comparison approach

“…Here, we adopted a hypothesis-driven variant of factorial model comparison (see van den Berg, Awh, & Ma, 2014), by generating 20 different models, each of which assumes that a change in task conditions will have a particular effect. The 20 models differ only in the specification of prior probability over parameters used in each task condition.…”