Getting the Within Estimator of Cross-Level Interactions in Multilevel Models with Pooled Cross-Sections: Why Country Dummies (Sometimes) Do Not Do the Job

Abstract

Multilevel models with persons nested in countries are increasingly popular in cross-country research. Recently, social scientists have started to analyze data with a three-level structure: persons at level 1, nested in year-specific country samples at level 2, nested in countries at level 3. By using a country fixed-effects estimator, or an alternative equivalent specification in a random-effects framework, this structure is increasingly used to estimate within-country effects in order to control for unobserved heterogeneity. For the main effects of country-level characteristics, such estimators have been shown to have desirable statistical properties. However, estimators of cross-level interactions in these models are not exhibiting these attractive properties: as algebraic transformations show, they are not independent of between-country variation and thus carry country-specific heterogeneity. Monte Carlo experiments consistently reveal the standard approaches to within estimation to provide biased estimates of cross-level interactions in the presence of an unobserved correlated moderator at the country level. To obtain an unbiased within-country estimator of a cross-level interaction, effect heterogeneity must be systematically controlled. By replicating a published analysis, we demonstrate the relevance of this extended country fixed-effects estimator in research practice. The intent of this article is to provide advice for multilevel practitioners, who will be increasingly confronted with the availability of pooled cross-sectional survey data.

Immigrant Sanctuary Policies and Crime-Reporting Behavior: A Multilevel Analysis of Reports of Crime Victimization to Law Enforcement, 1980 to 2004

“…The analyses presented here are not without limitations. First, although we leverage the longitudinal nature of our dataset and use a novel hybrid method to correctly estimate the "within-effect" component of our crosslevel interactions (Giesselmann and Schmidt-Catran 2019), which are more appropriate for drawing causal inferences (Allison 2009), our estimates are not impervious to sources of bias. We have striven to account for external environmental conditions that are demonstrated or theorized to be related to both help-seeking behavior (Baumer 2002;Felson et al 2002;Gutierrez and Kirk 2017;Xie and Baumer 2019b) and the passage of sanctuary policies Gonzalez et al 2017;Lyons et al 2013), but it is possible that some other unobserved time-varying condition has led us to over or underestimate the extent to which sanctuary policies condition the probability that Latino victims notify law enforcement.…”

“…For the sake of brevity, we focus on the interpretation of effects relevant to our specific research questions and corresponding hypotheses. Additionally, we concentrate on the within-effects (see the coefficients listed under the "MSA characteristics: within-effects" headings) given their suitability for causal inference (Giesselmann and Schmidt-Catran 2019). First, we find no evidence that sanctuary policy adoption, in general, is associated with crime-reporting behavior.…”

The nature of negative contact

“…The within level component is calculated by subtracting the time-variant scores in each year from the between level means (Fairbrother, 2013). Our models are thus group mean centered, as has been advocated for in the case of longitudinal multilevel models (Fairbrother & Martin, 2013;Giesselmann & Schmidt-Catran, 2018;Moller et al, 2009). We further included a variable for time on the within level to control for the possibility of simultaneous but unrelated and spurious time trends in hate crimes and in any of the independent variables (Fairbrother, 2013).…”

“…Second, using longitudinal data requires fewer assumptions about unobserved differences between municipalities being unimportant for the effects that are being studied (Giesselmann & Schmidt-Catran, 2018;Te Grotenhuis et al, 2015).…”

“…Second, our longitudinal research on the relationship between changes over time within places in racial composition and changes in the number of committed hate crimes, requires fewer assumptions about unobserved differences between places (Giesselmann & Schmidt-Catran, 2018;Te Grotenhuis et al, 2015). Using data from the FBI and the U.S. Census Bureau, covering a timespan from 1990 to 2014 and 3.570 places, we estimated multilevel models with within-place differences between years (level 1), nested in places as clusters (level 2).…”