Logical Inconsistency in EI‐Based Second‐Stage Regressions

Abstract

The statistical procedure EI-R, in which point estimates produced by the King (1997) ecological inference technique are used as dependent variables in a linear regression, can be logically inconsistent insofar as the assumptions necessary to support EI-R's first stage (ecological inference via King's technique) can be incompatible with the assumptions supporting its second stage (linear regression). In light of this problem, we develop a specification test for logical consistency of EI-R and describe options available to a researcher who confronts test rejection. We then apply our test to the implementation of EI-R in Burden and Kimball's (1998) study of ticket splitting and find that this implementation is logically inconsistent. In correcting for this problem we show that Burden and Kimball's substantive results are artifacts of a self-contradictory statistical technique. I t is becoming increasingly common for researchers to use point estimates produced by the King (1997) ecological inference technique as dependent variables in second-stage linear regressions. This two-stage statistical procedure, which Herron and Shotts (2003b) call EI-R, was initially proposed by King (1997), who has said that "[EI-R] has enormous potential for uncovering new information" (1997, 279). Burden and Kimball (1998), for example, use King-based estimates of district-level ticket-splitting rates as dependent variables in secondstage regressions which examine why individuals cast split-ticket ballots. Other examples of EI-R and variants of this two-stage statistical technique can be found in logically inconsistent insofar as the assumptions necessary to support the procedure's first stage (ecological inference via King's method) can be incompatible with the assumptions supporting its second stage (linear regression). In contrast, we say that an application of EI-R is logically consistent when its first-stage and second-stage assumptions are not mutually contradictory.The matter of EI-R's being logically consistent merits attention because EI-R has two stages, unlike a regular regression that has only a single stage. Whenever a researcher employs a two-stage statistical procedure, she must simultaneously adopt assumptions that support both stages. Previous work has not sought to determine whether this is possible in the case of EI-R. Indeed, to the best of our knowledge all published and working-paper implementations of EI-R assume that the procedure's two stages are inherently compatible.Nonetheless, we show that the assumptions necessary to support EI-R can be self-contradictory. This result follows from the fact that standard implementations of King's technique to an ecological dataset (the first stage of EI-R) assume that there is no aggregation

The Limits of Ecological Inference: The Case of Split‐Ticket Voting

“…Since EI generally does not outperform OLS (Cho 1998), it is difficult to justify the additional overhead. EI does supply district‐level estimates, but these estimates do not possess desirable statistical properties (Herron and Shotts 2003, 2004). There are instances when one needs to make ecological inferences, and so one will choose to use an ecological inference model such as EI.…”

“…Each stage of their estimation beginning with the conceptualization of the problem, however, has difficulties. We do not examine the last stage of their estimation closely, but note that Herron and Shotts (2003, 2004) and McCue (2001) have scrutinized the validity of using point estimates generated by EI as dependent variables in a second‐stage linear regression, the exact process by which Burden and Kimball arrived at their final estimation. The analysis by Herron and Shotts shows that this process may yield inconsistent and attenuated estimates, but worse, these estimates may suffer from sign reversal and augmentation bias.…”

“…This ostensible “advantage” of EI, however, is largely illusory, as these district estimates are not consistent and may lead to erroneous inferences. Some of the grave risks entailed in using EI district estimates as dependent variables in other models have been chronicled by Herron and Shotts (2003, 2004) and McCue (2001).…”

Ecological Inference Techniques: An Empirical Evaluation Using Data Describing Gender and Voter Turnout at New Zealand Elections, 1893–1919

“…Future work is also needed to identify the effect of aggregation bias on estimates, and to inform the use of estimates in EI regressions (Herron and Schotts, 2004). Further areas for theoretical development include extending existing work on EI to methods that are based on r × c tables Rosen et al, 2001;Imai et al, 2009) to model interactions between factors such as gender, class and race.…”

“…First studied by sociologists in the 1950s (Robinson, 1950;Duncan and Davis, 1953;Goodman, 1953), in the last decade or so there has been resurgent interest in EI by political methodologists and statisticians (Achen and Shivley, 1995;King, 1997;King et al, 2004;Wakefield, 2004a, b). Much of the recent research has focused on the development of new parametric models and led to debates over the validity of methods proposed and their utility (Freedman et al, 1991;Cho and Gaines, 2004;Herron and Schotts, 2004). Very recently Imai et al (2008) have placed EI within the theoretical framework of incomplete (missing) data.…”