Using priors to formalize theory: Optimal attention and the generalized context model

Abstract

Formal models in psychology are used to make theoretical ideas precise and allow them to be evaluated quantitatively against data. We focus on one important-but underused and incorrectly maligned-method for building theoretical assumptions into formal models, offered by the Bayesian statistical approach. This method involves capturing theoretical assumptions about the psychological variables in models by placing informative prior distributions on the parameters representing those variables. We demonstrate this approach of casting basic theoretical assumptions in an informative prior by considering a case study that involves the generalized context model (GCM) of category learning. We capture existing theorizing about the optimal allocation of attention in an informative prior distribution to yield a model that is higher in psychological content and lower in complexity than the standard implementation. We also highlight that formalizing psychological theory within an informative prior distribution allows standard Bayesian model selection methods to be applied without concerns about the sensitivity of results to the prior. We then use Bayesian model selection to test the theoretical assumptions about optimal allocation formalized in the prior. We argue that the general approach of using psychological theory to guide the specification of informative prior distributions is widely applicable and should be routinely used in psychological modeling.

Bayesian data analysis for newcomers

“…The sensitivity analysis helps put boundaries on the conclusions when there is uncertainty in choice of prior. Various authors emphasize that the influence of the prior distribution in Bayesian model comparison is an importantly positive feature because it forces the theorist to acknowledge that the prior distribution on the parameters is a central aspect of the expression of the theory (Vanpaemel, 2010;Vanpaemel & Lee, 2012). For example, a theory that predicts that forgetting should occur within some small range of decay rates is quite different that a theory that predicts the decay rate could be anything.…”

“…The resulting posterior distributions from the previous data act as the prior distributions for the model comparison (Kruschke, 2015, Section 10.6.1, p. 294). For fur-ther reading about Bayesian model comparison and setting useful priors within models, see examples in Kary, Taylor, and Donkin (2016) and in Vanpaemel and Lee (2012). The setting of prior probabilities on the model index is also important but less often considered.…”

The Bayesian New Statistics: Hypothesis testing, estimation, meta-analysis, and power analysis from a Bayesian perspective

“…First, the BF is extremely sensitive to the choice of prior distribution for the alternative hypothesis. Therefore in realistic application it is important to use a theoretically meaningful and informed distribution for both the prior and alternative hypotheses, not merely generic defaults, and it is important to check that the BF does not change much if the prior distributions are changed in reasonable ways (e.g., Dienes, 2014Dienes, , 2016Kruschke, 2011a;Lee & Wagenmakers, 2014;Vanpaemel & Lee, 2012). Second, the BF does not indicate the posterior odds, and users must remember to take into account the prior odds of the hypotheses.…”

“…The alternative-hypothesis prior on θ could be generically vague and set by default, or the alternative-hypothesis prior on θ could be meaningfully informed by previous data or theory. Although we will illustrate Bayesian null hypothesis testing by using a default alternative prior, in applied practice it is important to use a meaningfully informed alternative prior (Dienes, 2014;Kruschke, 2011a;Vanpaemel and Lee, 2012). Figure 6 illustrates the parameter space for Bayesian null hypothesis testing.…”

“…For these three generic illustrations we computed BFs using the BayesFactor R package (Morey et al, 2015) with default alternative hypothesis rscale=0.707. In applied research, it would be important to use meaningfully informed alternative priors, which would result in different values for the BF's (Dienes, 2014;Kruschke, 2011a;Vanpaemel and Lee, 2012).…”