Bayesian multiple comparisons of simply ordered means using priors with a point mass

Nashimoto, Kane; Wright, F. T.

doi:10.1016/j.csda.2008.05.015

Cited by 6 publications

(2 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The potential utility of using formal decision theoretic or model selection approaches in place of a more traditional multiple comparisons procedure is just one of several potential solutions. Other solutions have been suggested in the context of Bayesian models (e.g., Li & Shang, 2015;Nashimoto & Wright, 2008;Shang et al, 2008;Westfall et al, 1997), but as noted previously, they introduce additional concerns. We emphasize that our goal is to lend insight into whether or not HB models are immune to multiplicity issues -they are notand to offer PPI as a tool for evaluating when alternative modelling approaches or adjustment procedures should be explored.…”

Section: Other Considerationsmentioning

confidence: 99%

“…The pooling property essentially 'pulls' each θ k towards the global mean, reducing differences among the levels (Gelman & Hill, 2007;Gelman, Hwang & Vehtari 2014). Others, however, suggest modifications to HB models to account for multiplicity (e.g., Li & Shang, 2015;Nashimoto & Wright, 2008;Shang, Cavanaugh, & Wright, 2008;Westfall et al, 1997), but the approaches can greatly increase the complexity of the model, require specification of informative priors that may not be supported by existing information or require an a priori ranking of the group-level means, which may be difficult to determine in practice. Thus, the suggestion that partial pooling diminishes the need to adjust for multiplicity is attractive but has not be rigorously tested against the types of 'messy' data that ecologists routinely work with.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Should we be concerned about multiple comparisons in hierarchical Bayesian models?

et al. 2019

View full text Add to dashboard Cite

Ecologists increasingly use hierarchical Bayesian (HB) models to estimate group‐level parameters that vary by, for example, species, treatment level, habitat type or other factors. Group‐level parameters may be compared to infer differences among levels. We would conclude a non‐zero pairwise difference, separately, for each pair in the group, when the respective 95% credible interval excludes zero. Classical procedures suggest that the rejection procedure should be adjusted to control the family‐wise error rate (FWER) for a family of differences. Adjustments for FWER have been considered unnecessary in HB models due to partial pooling whereby increased pooling strength – group‐level parameters become more alike – could lead to decreased rejection rates (Type I error, FWER, or Power) and increased false acceptance rates (Type 2 error and its family‐wise analogue). To address this, we conducted a simulation experiment with factors of sample size, group size, balance (missingness), overall mean and ratio of within‐ to between‐group variances, resulting in 2016 factor‐level combinations (‘scenarios’), replicated 100 times, producing 201,600 pseudo datasets analysed in a Bayesian framework. We evaluated the results in the context of a new partial pooling index (PPI), which we show is also applicable to more complex model structures based on four real‐data examples. Simulation results confirm intuition that rejection rates (false and true) decrease and false acceptance rates increase with increasing PPI or pooling strength (scenario‐level R2 = 0.81–0.97). The relationship with PPI differed greatly for balanced versus unbalanced designs and was affected by group size, especially for family‐wise errors. Critically, an HB model does not guarantee that the FWER will follow a set significance level (α); for example, even minor imbalance can lead to FWER > α for weak to moderate pooling. These results are confirmed by the real‐data examples, suggesting that ecologists need to consider FWER when applying HB models, especially for large group sizes or incomplete datasets. Contrary to current thought, HB models are not immune to issues of multiplicity, and our proposed PPI offers a method for evaluating if a particular HB analysis is likely to produce FWER ≤ α (no adjustment or alternative solution required).

show abstract

Section: Other Considerationsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Should we be concerned about multiple comparisons in hierarchical Bayesian models?

et al. 2019

View full text Add to dashboard Cite

show abstract

Sharp bounds on distribution functions and expectations of mixtures of ordered families of distributions

Rychlik

2018

TEST

View full text Add to dashboard Cite

We compare two mixtures of arbitrary collection of stochastically ordered distribution functions with respect to fixed mixing distributions. Under the assumption that the first mixture distribution is known, we establish optimal lower and upper bounds on the latter mixture distribution function and present single families of ordered distributions which attain the bounds uniformly at all real arguments. Furthermore, we determine sharp upper and lower bounds on the differences between the expectations of the mixtures expressed in various scale units. General results are illustrated by several examples.

show abstract

Variation of the expectation and quantiles of the mixture of ordered distributions under imprecise choice of prior

Rychlik

2021

Statistics

View full text Add to dashboard Cite

Bayesian multiple comparisons of simply ordered means using priors with a point mass

Cited by 6 publications

References 22 publications

Should we be concerned about multiple comparisons in hierarchical Bayesian models?

Should we be concerned about multiple comparisons in hierarchical Bayesian models?

Sharp bounds on distribution functions and expectations of mixtures of ordered families of distributions

Variation of the expectation and quantiles of the mixture of ordered distributions under imprecise choice of prior

Contact Info

Product

Resources

About