What You Can—and Can’t—Do With Three-Wave Panel Data

Abstract

The recent change in the general social survey (GSS) to a rotating panel design is a landmark development for social scientists. Sociological methodologists have argued that fixed-effects (FE) models are generally the best starting point for analyzing panel data because they allow analysts to control for unobserved time-constant heterogeneity. We review these treatments and demonstrate the advantages of FE models in the context of the GSS. We also show, however, that FE models have two rarely tested assumptions that can seriously bias parameter estimates when violated. We provide simple tests for these assumptions. We further demonstrate that FE models are extremely sensitive to the correct specification of temporal lags. We provide a simulation and a proof to show that the use of incorrect lags in FE models can lead to coefficients that are the opposite sign of the true parameter values.

How to Deal With Reverse Causality Using Panel Data? Recommendations for Researchers Based on a Simulation Study

“…Following Vaisey and Miles (2017), we distinguish between three worlds that differ with respect to the timing of causal effects. In the contemporaneous world, has an effect on .…”

“…In contrast, the lagged first-difference (LFD) model accounts for both time-invariant unobserved heterogeneity and reverse causality, as it relaxes the strict exogeneity assumption by requiring only sequential exogeneity. As shown by Vaisey and Miles (2017), however, the LFD model only provides unbiased estimates if the effect of X on Y is indeed fully lagged, thus being prone to specification error. Finally, the AB and ML-SEM models also promise to perform well in case of time-invariant unobserved heterogeneity and/or reverse causality, the latter of which is achieved by assuming sequential rather than strict exogeneity.…”

“…A key reason for the popularity of panel models is that they allow to exploit change within units over time (e.g., individual change) to eliminate unobserved time-invariant heterogeneity, which considerably reduces the risk of confounding (Allison 2009;Halaby 2004;Wooldridge 2010). A second reason why researchers frequently turn to panel data is that they expect them to determine causal order (Vaisey and Miles 2017). Reconsidering the above-mentioned research questions, one might ask: Does feminization of occupations reduce pay?…”

“…However, these earlier simulations do not consider the problem raised by Vaisey and Miles (2017), i.e., that inference based on first-difference models is prone to bias due to misspecification of temporal lags. 7 Hence, like for RE, FE, and AB models, it remains open whether the ML-SEM method for cross-lagged panel models with fixed effects is sensitive to the correct specification of temporal lags.…”

“…Unfortunately, though, as Vaisey and Miles (2017) in the process under study. This risk of specification error highlights the need for precise theorizing regarding the actual lag structure of the causal process under investigation.…”