Estimation and Inference for Very Large Linear Mixed Effects Models

Gao, Katelyn; Owen, Art B.

doi:10.5705/ss.202018.0029

Cited by 11 publications

(18 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…See also Section 1 of Gao & Owen (2019) for related discussion and references. Depending on the data design, the precision matrices of the Gaussian distributions involved may be sparse, in which case black-box sparse linear algebra algorithms can be used for the matrix operations in order to reduce the computational cost.…”

Section: Introductionmentioning

confidence: 99%

Scalable inference for crossed random effects models

2019

View full text Add to dashboard Cite

Summary We develop methodology and complexity theory for Markov chain Monte Carlo algorithms used in inference for crossed random effects models in modern analysis of variance. We consider a plain Gibbs sampler and propose a simple modification, referred to as a collapsed Gibbs sampler. Under some balancedness conditions on the data designs and assuming that precision hyperparameters are known, we demonstrate that the plain Gibbs sampler is not scalable, in the sense that its complexity is worse than proportional to the number of parameters and data, but the collapsed Gibbs sampler is scalable. In simulated and real datasets we show that the explicit convergence rates predicted by our theory closely match the computable, but nonexplicit rates in cases where the design assumptions are violated. We also show empirically that the collapsed Gibbs sampler extended to sample precision hyperparameters significantly outperforms alternative state-of-the-art algorithms.

show abstract

Section: Introductionmentioning

confidence: 99%

Scalable inference for crossed random effects models

2019

View full text Add to dashboard Cite

show abstract

“…To extend and see the limits of multigrid decomposition, we consider the following simple extension, which just replaces the intercept term µ with a linear combination of p covariates with a fixed coefficient vector. Previously, Gao and Owen (2019) attempted to tackle the computational efficiency of this model (3.1). But their results give loose bounds while requiring mild conditions.…”

Section: Linear Mixed Effects Modelsmentioning

confidence: 99%

On Gibbs Sampling for Structured Bayesian Models Discussion of paper by Zanella and Roberts

Yang¹,

Liu²

2021

Preprint

View full text Add to dashboard Cite

We congratulate Professors Giacomo Zanella and Gareth Roberts for their path-breaking work in analyzing Gibbs sampling algorithms for a class of highly practical Bayesian hierarchical models. Together with their previous work, Papaspiliopoulos and Roberts (2003) and Papaspiliopoulos et al. (2020), their multigrid decomposition strategy elegantly reduces a high-dimensional Gibbs sampling algorithm to independent low-dimensional components so that the convergence rate of the Gibbs sampler can be determined analytically.These are extremely interesting and encouraging results. Throughout of the article, we will refer to this work of Zanella and Roberts (2021) as "Z&R" for simplicity.The multigrid decomposition serves a central role in the whole theory established in the aforementioned series of papers. An intuition behind this decomposition is that lower-level mean statistics are sufficient for posterior inference on upper-level parameters, with lowerlevel parameters practically marginalized out. For example, Papaspiliopoulos and Roberts * This paper was done while Xiaodong Yang was working as a remote summer undergraduate at Professor Jun S. Liu's lab at Harvard, in the summer of 2021.

show abstract

“…Not only can this present computational challenges, but these models only guarantee valid statistical inference when the specific model for dependence posited is correct, unlike the very general forms of dependence for which cluster-robust bootstraps are valid (Cameron & Miller, 2015;Owen & Eckles, 2012). In the case of multiple sources of dependence (i.e., crossed random effects), fitting crossed random effects models remains especially difficult to parallelize (Gao & Owen, 2016), but bootstrap methods remain mildly conservative (McCullagh, 2000;Owen, 2007;Owen & Eckles, 2012).…”

Section: Dependent Datamentioning

confidence: 99%

Bootstrap Thompson Sampling and Sequential Decision Problems in the Behavioral Sciences

Eckles

Kaptein

2019

Sage Open

View full text Add to dashboard Cite

Behavioral scientists are increasingly able to conduct randomized experiments in settings that enable rapidly updating probabilities of assignment to treatments (i.e., arms). Thus, many behavioral science experiments can be usefully formulated as sequential decision problems. This article reviews versions of the multiarmed bandit problem with an emphasis on behavioral science applications. One popular method for such problems is Thompson sampling, which is appealing for randomizing assignment and being asymptoticly consistent in selecting the best arm. Here, we show the utility of bootstrap Thompson sampling (BTS), which replaces the posterior distribution with the bootstrap distribution. This often has computational and practical advantages. We illustrate its robustness to model misspecification, which is a common concern in behavioral science applications. We show how BTS can be readily adapted to be robust to dependent data, such as repeated observations of the same units, which is common in behavioral science applications. We use simulations to illustrate parametric Thompson sampling and BTS for Bernoulli bandits, factorial Gaussian bandits, and bandits with repeated observations of the same units.

show abstract

Estimation and Inference for Very Large Linear Mixed Effects Models

Cited by 11 publications

References 23 publications

Scalable inference for crossed random effects models

Scalable inference for crossed random effects models

On Gibbs Sampling for Structured Bayesian Models Discussion of paper by Zanella and Roberts

Bootstrap Thompson Sampling and Sequential Decision Problems in the Behavioral Sciences

Contact Info

Product

Resources

About