Quantifying uncertainty in transdimensional Markov chain Monte Carlo using discrete Markov models

Heck, Daniel W.; Overstall, Antony M.; Gronau, Quentin Frederik; Wagenmakers, Eric‐Jan

doi:10.1007/s11222-018-9828-0

Cited by 17 publications

(13 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Mitchell & Beauchamp, 1988) and 2) a diffuse "slab" component surrounding zero. A central aspect of this approaches is the addition of an indicator variable (Kuo & Mallick, 1998), which in essence allows for switching between the mixture components (i.e., transdimensional MCMC, Heck, Overstall, Gronau, & Wagenmakers, 2018). The proportion of MCMC samples spent in each component can then be used to approximate the respective posterior model probabilities or the marginal Bayes factors.…”

Section: Random Intercept Modelmentioning

confidence: 99%

Beneath the surface: Unearthing within-person variability and mean relations with Bayesian mixed models.

Williams¹,

Mulder²,

Rouder³

et al. 2021

Psychological Methods

View full text Add to dashboard Cite

Mixed-effects models are becoming common in psychological science. Although they have many desirable features, there is still untapped potential. It is customary to view homogeneous variance as an assumption to satisfy. We argue to move beyond that perspective, and to view modeling within-person variance as an opportunity to gain a richer understanding of psychological processes. The technique to do so is based on the mixed-effects location scale model that can simultaneously estimate mixed-effects sub-models to both the mean (location) and within-person variance (scale). We develop a framework that goes beyond assessing the sub-models in isolation of one another and introduce a novel Bayesian hypothesis test for mean-variance correlations in the distribution of random effects. We first present a motivating example, which makes clear how the model can characterize mean-variance relations. We then apply the method to reaction times gathered from two cognitive inhibition tasks. We find there are more individual differences in the within-person variance than the mean structure, as well as a complex web of structural mean-variance relations. This stands in contrast to the dominant view of within-person variance (i.e., "noise"). The results also point towards paradoxical within-person, as opposed to between-person, effects: several people had slower and less variable incongruent responses. This contradicts the typical pattern, wherein larger means tend to be associated with more variability. We conclude with future directions, spanning from methodological to theoretical inquires, that can be answered with the presented methodology.

show abstract

Section: Random Intercept Modelmentioning

confidence: 99%

Beneath the surface: Unearthing within-person variability and mean relations with Bayesian mixed models.

Williams¹,

Mulder²,

Rouder³

et al. 2021

Psychological Methods

View full text Add to dashboard Cite

show abstract

“…A central aspect of this approach is the addition of a binary indicator, which in essence allows for switching between the two mixture components (i.e., transdimensional MCMC; Heck, Overstall, Gronau, & Wagenmakers, 2018). The proportion of MCMC samples spent in each component can then be used to approximate the respective posterior model probabilities.…”

Section: Spike and Slab Prior Distributionmentioning

confidence: 99%

Putting the Individual into Reliability: Bayesian Testing of Homogeneous Within-Person Variance in Hierarchical Models

Williams¹,

Martin²,

Rast³

2019

Preprint

View full text Add to dashboard Cite

Measurement reliability is a fundamental concept in psychology. It is traditionally considered a stable property of a questionnaire, measurement device, or experimental task. Although intraclass correlation coefficients (ICC) are often used to assess reliability in repeated measure designs, their descriptive nature depends upon the assumption of a commonwithin-person variance.This work focuses on the presumption that each individual is adequately described by the average within-person variance in hierarchical models. And thus whether reliability generalizes to the individual level, which leads directly into the notion of individually varying ICCs. In particular, we introduce a novel approach, using the Bayes factor, wherein a researcher can directly test for homogeneous within-person variance in hierarchical models. Additionally, we introduce a membership model that allows for classifying which (and how many) individuals belong to the common variance model. The utility of our methodology is demonstrated on cognitive inhibition tasks. We find that heterogeneous within-person variance is a defining feature of these tasks, and in one case, the ratio between the largest to smallest within-person variance exceeded 20. This translates into a 10 fold difference in person-specific reliability! We also find that few individuals belong to the common variance model, and thus traditional reliability indices are potentially masking important individual variation. We discuss the implications of our findings and possible future directions. The methods are implemented in the R package vICC.

show abstract

“…For adaptive MCMC, Atchadé () provides estimators for the asymptotic variance of the Monte Carlo estimator. Heck, Overstall, Gronau, and Wagenmakers () provide uncertainty quantification for trans‐dimensional MCMC methods. However, the literature for these processes is not as rich as traditional MCMC, and would benefit from further work.…”

Section: Extensionsmentioning

confidence: 99%

Analyzing Markov chain Monte Carlo output

Vats

Robertson

Flegal

et al. 2020

WIREs Computational Stats

View full text Add to dashboard Cite

Markov chain Monte Carlo (MCMC) is a sampling-based method for estimating features of probability distributions. MCMC methods produce a serially correlated, yet representative, sample from the desired distribution. As such it can be difficult to assess when the MCMC method is producing reliable results. We present some fundamental methods for ensuring a reliable simulation experiment. In particular, we present a workflow for output analysis in MCMC providing estimators, approximate sampling distributions, stopping rules, and visualization tools. This article is categorized under: Statistical Models > Bayesian Models Statistical and Graphical Methods of Data Analysis > Markov Chain Monte

show abstract

Quantifying uncertainty in transdimensional Markov chain Monte Carlo using discrete Markov models

Cited by 17 publications

References 36 publications

Beneath the surface: Unearthing within-person variability and mean relations with Bayesian mixed models.

Beneath the surface: Unearthing within-person variability and mean relations with Bayesian mixed models.

Putting the Individual into Reliability: Bayesian Testing of Homogeneous Within-Person Variance in Hierarchical Models

Analyzing Markov chain Monte Carlo output

Contact Info

Product

Resources

About