Exchangeability, Correlation, and Bayes' Effect

O'Neill, Ben

doi:10.1111/j.1751-5823.2008.00059.x

Cited by 25 publications

(13 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Third, to quantify the danger that strong evidence in favor of our alternative hypotheses could arise by chance from testing 3528 estimated cortical sources, permutation tests for each hypothesis were performed (10,000 iterations, randomly shuffling the order of conditions 3528 times within participants). Permutation tests have been used extensively to control type 1 errors when analyzing neuroimaging data from a frequentist statistical perspective (Maris and Oostenveld, 2007;Winkler et al, 2014) and conceptually tie directly to the Bayesian assumption of exchangeability (O'Neill, 2009). The 95th percentiles of the empirical distributions of Bayes factor values arising from these permutation tests were far below (ssVEP, 0.54; alpha, 1.079) the minimum Bayes factor of 3:1 in favor of the alternative hypothesis that we adopt in displaying or discussing statistical comparisons.…”

Section: Discussionmentioning

confidence: 99%

Aversive Conditioning of Spatial Position Sharpens Neural Population-Level Tuning in Visual Cortex and Selectively Alters Alpha-Band Activity

Friedl

Keil

2021

J. Neurosci.

View full text Add to dashboard Cite

Processing capabilities for many low-level visual features are experientially malleable, aiding sighted organisms in adapting to dynamic environments. Explicit instructions to attend a specific visual field location influence retinotopic visuocortical activity, amplifying responses to stimuli appearing at cued spatial positions. It remains undetermined both how such prioritization affects surrounding nonprioritized locations, and if a given retinotopic spatial position can attain enhanced cortical representation through experience rather than instruction. The current report examined visuocortical response changes as human observers (N = 51, 19 male) learned, through differential classical conditioning, to associate specific screen locations with aversive outcomes. Using dense-array EEG and pupillometry, we tested the preregistered hypotheses of either sharpening or generalization around an aversively associated location following a single conditioning session. Competing hypotheses tested whether mean response changes would take the form of a Gaussian (generalization) or difference-of-Gaussian (sharpening) distribution over spatial positions, peaking at the viewing location paired with a noxious noise. Occipital 15 Hz steady-state visual evoked potential responses were selectively heightened when viewing aversively paired locations and displayed a nonlinear, difference-of-Gaussian profile across neighboring locations, consistent with suppressive surround modulation of nonprioritized positions. Measures of alpha-band (8-12 Hz) activity were differentially altered in anterior versus posterior locations, while pupil diameter exhibited selectively heightened responses to noise-paired locations but did not evince differences across the nonpaired locations. These results indicate that visuocortical spatial representations are sharpened in response to location-specific aversive conditioning, while top-down influences indexed by alpha-power reduction exhibit posterior generalization and anterior sharpening.

show abstract

Section: Discussionmentioning

confidence: 99%

Aversive Conditioning of Spatial Position Sharpens Neural Population-Level Tuning in Visual Cortex and Selectively Alters Alpha-Band Activity

Friedl

Keil

2021

J. Neurosci.

View full text Add to dashboard Cite

show abstract

“…An experimental concept will be incorrect when one or more of its exchangeability judgments violate reality. The constituent assumptions can often be tested independently of the model; e.g., through correlation analysis and other types of data checks (57,74,75). In our example, if the frequency estimator k/N increases in the long run (e.g., if the proportion N 1 /N 0 increases), then the event set is not likely to be exchangeable, and the experimental concept of test 1 needs to be rethought to allow for this secular time dependence.…”

Section: Discussionmentioning

confidence: 99%

Testing for ontological errors in probabilistic forecasting models of natural systems

Marzocchi

Jordan

2014

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

Probabilistic forecasting models describe the aleatory variability of natural systems as well as our epistemic uncertainty about how the systems work. Testing a model against observations exposes ontological errors in the representation of a system and its uncertainties. We clarify several conceptual issues regarding the testing of probabilistic forecasting models for ontological errors: the ambiguity of the aleatory/epistemic dichotomy, the quantification of uncertainties as degrees of belief, the interplay between Bayesian and frequentist methods, and the scientific pathway for capturing predictability. We show that testability of the ontological null hypothesis derives from an experimental concept, external to the model, that identifies collections of data, observed and not yet observed, that are judged to be exchangeable when conditioned on a set of explanatory variables. These conditional exchangeability judgments specify observations with well-defined frequencies. Any model predicting these behaviors can thus be tested for ontological error by frequentist methods; e.g., using P values. In the forecasting problem, prior predictive model checking, rather than posterior predictive checking, is desirable because it provides more severe tests. We illustrate experimental concepts using examples from probabilistic seismic hazard analysis. Severe testing of a model under an appropriate set of experimental concepts is the key to model validation, in which we seek to know whether a model replicates the data-generating process well enough to be sufficiently reliable for some useful purpose, such as long-term seismic forecasting. Pessimistic views of system predictability fail to recognize the power of this methodology in separating predictable behaviors from those that are not.system science | Bayesian statistics | significance testing | subjective probability | expert opinion S cience is rooted in the concept that a model can be tested against observations and rejected when necessary (1). However, the problem of model testing becomes formidable when we consider natural systems. Owing to their scale, complexity, and openness to interactions within a larger environment, most natural systems cannot be replicated in the laboratory, and direct observations of their inner workings are always inadequate. These difficulties raise serious questions about the meaning and feasibility of "model verification" and "model validation" (2), and have led to the pessimistic view that "the outcome of natural processes in general cannot be accurately predicted by mathematical models" (3).Uncertainties in the formal representation of natural systems imply that the forecasting of emergent phenomena such as natural hazards must be based on probabilistic rather than deterministic modeling. The ontological framework for most probabilistic forecasting models comprises two types of uncertainty: an aleatory variability that describes the randomness of the system, and an epistemic uncertainty that characterizes our lack of knowledge about the sy...

show abstract

“…Whether frequentist or Bayesian, the essential statistical nature of random effects stems from their specification as arising from a common probability distribution whose parameters, to be estimated in some manner, often, but not always, include just a single variance parameter (Kutner et al 2004, Gamerman and Lopes 2006, Gelman and Hill 2007, Ramsey and Schafer 2013, Gelman et al 2014). In the context of random effects, the units or group levels, i.e., experimental units, observational units, individuals, subjects, etc., such as a subset of trees randomly selected from a multi‐hectare plot, are often viewed as exchangeable (Draper et al 1993, O'Neill 2009). In particular, the observed units (e.g., trees) are typically treated as conditionally independent, arising from a common probability distribution described by (conditional on), for example, variance and/or covariance parameters that quantify variability among the units.…”

Section: A Bayesian Perspective On Fixed Vs Random Effectsmentioning

confidence: 99%

Ensuring identifiability in hierarchical mixed effects Bayesian models

Ogle

Barber

2020

Ecological Applications

View full text Add to dashboard Cite

Ecologists are increasingly familiar with Bayesian statistical modeling and its associated Markov chain Monte Carlo (MCMC) methodology to infer about or to discover interesting effects in data. The complexity of ecological data often suggests implementation of (statistical) models with a commensurately rich structure of effects, including crossed or nested (i.e., hierarchical or multi-level) structures of fixed and/or random effects. Yet, our experience suggests that most ecologists are not familiar with subtle but important problems that often arise with such models and with their implementation in popular software. Of foremost consideration for us is the notion of effect identifiability, which generally concerns how well data, models, or implementation approaches inform about, i.e., identify, quantities of interest. In this paper, we focus on implementation pitfalls that potentially misinform subsequent inference, despite otherwise informative data and models. We illustrate the aforementioned issues using random effects regressions on synthetic data. We show how to diagnose identifiability issues and how to remediate these issues with model reparameterization and computational and/or coding practices in popular software, with a focus on JAGS, OpenBUGS, and Stan. We also show how these solutions can be extended to more complex models involving multiple groups of nested, crossed, additive, or multiplicative effects, for models involving random and/or fixed effects. Finally, we provide example code (JAGS/OpenBUGS and Stan) that practitioners can modify and use for their own applications.

show abstract

Exchangeability, Correlation, and Bayes' Effect

Cited by 25 publications

References 11 publications

Aversive Conditioning of Spatial Position Sharpens Neural Population-Level Tuning in Visual Cortex and Selectively Alters Alpha-Band Activity

Aversive Conditioning of Spatial Position Sharpens Neural Population-Level Tuning in Visual Cortex and Selectively Alters Alpha-Band Activity

Testing for ontological errors in probabilistic forecasting models of natural systems

Ensuring identifiability in hierarchical mixed effects Bayesian models

Contact Info

Product

Resources

About