Understanding, Choosing, and Unifying Multilevel and Fixed Effect Approaches

Abstract

When working with grouped data, investigators may choose between “fixed effects” models (FE) with specialized (e.g., cluster-robust) standard errors, or “multilevel models” (MLMs) employing “random effects.” We review the claims given in published works regarding this choice, then clarify how these approaches work and compare by showing that: (i) random effects employed in MLMs are simply “regularized” fixed effects; (ii) unmodified MLMs are consequently susceptible to bias—but there is a longstanding remedy; and (iii) the “default” MLM standard errors rely on narrow assumptions that can lead to undercoverage in many settings. Our review of over 100 papers using MLM in political science, education, and sociology show that these “known” concerns have been widely ignored in practice. We describe how to debias MLM’s coefficient estimates, and provide an option to more flexibly estimate their standard errors. Most illuminating, once MLMs are adjusted in these two ways the point estimate and standard error for the target coefficient are exactly equal to those of the analogous FE model with cluster-robust standard errors. For investigators working with observational data and who are interested only in inference on the target coefficient, either approach is equally appropriate and preferable to uncorrected MLM.

A practical guide to selecting and blending approaches for clustered data: Clustered errors, multilevel models, and fixed-effect models.

“…However, this property of fixed-effect models can be recreated in a multilevel model without sacrificing the ability to estimate effects of specific predictors in the between-cluster submodel (A. Bell & Jones, 2015; Dieleman & Templin, 2014; Hazlett & Wainstein, 2022; McNeish & Kelley, 2019). In a multilevel model, this can be accomplished by (a) cluster-mean centering all predictors in the within-cluster submodel and (b) including the cluster means of all within-cluster predictors as predictors of the intercept in the between-cluster submodel.…”

“…Specifying a multilevel model this way artificially forces the two submodels to be orthogonal (Hamaker & Muthén, 2020; Hazlett & Wainstein, 2022), mimicking the process in fixed-effect models but by a different mechanism. A fixed-effect model explains all between-cluster variance with cluster affiliation dummies, making between-cluster information orthogonal to within-cluster information (i.e., if unexplained between-cluster variance is zero, any covariance is also necessarily zero).…”

“…This specification has been referred to as the within-between specification (A. Bell & Jones, 2015;Dieleman & Templin, 2014;McNeish & Kelley, 2019) or the biascorrected multilevel model (Hazlett & Wainstein, 2022) and is related to the Mundlak (1978) and Chamberlain (1982) devices from econometrics, where it is better known as the correlated random effects approach (Schunck, 2013;Wooldridge, 2010).…”

Using Multitask Gaussian Processes to estimate the effect of a targeted effort to remove firearms

“…We could instead model random effects using a hierarchical prior. See Hazlett and Wainstein (2020) for discussion.…”

Change-Point Detection and Regularization in Time Series Cross-Sectional Data Analysis

“…We first introduce the hidden Markov Bayesian bridge model (HMBB), which combines a Bayesian bridge model for parameter regularization with a hidden Markov model for multiple change-point detection. In this paper, we present HMBB in the context of TSCS data because TSCS data have been the core of dynamic model development in political science literature (Beck 2001;Beck et al 1993;Beck and Katz 1995;Beck and Katz 2011;Box-Steffensmeier et al 2014;Brandt and Freeman 2006;Hazlett and Wainstein 2022;Imai and Kim 2021;Pang, Liu, and Xu 2022;Western and Kleykamp 2004;Wucherpfennig et al 2021).…”