How Multiple Imputation Makes a Difference

Abstract

Edited by R. Michael AlvarezPolitical scientists increasingly recognize that multiple imputation represents a superior strategy for analyzing missing data to the widely used method of listwise deletion. However, there has been little systematic investigation of how multiple imputation affects existing empirical knowledge in the discipline. This article presents the first large-scale examination of the empirical effects of substituting multiple imputation for listwise deletion in political science. The examination focuses on research in the major subfield of comparative and international political economy (CIPE) as an illustrative example. Specifically, I use multiple imputation to reanalyze the results of almost every quantitative CIPE study published during a recent fiveyear period in International Organization and World Politics, two of the leading subfield journals in CIPE. The outcome is striking: in almost half of the studies, key results "disappear" (by conventional statistical standards) when reanalyzed.

Which tests not witch hunts: a diagnostic approach for conducting replication research

“…Researchers may choose to drop observations with missing values, assign all missing values the same value (based on an assumption, for example, about why a response was not given), impute missing values using variable means, or use other imputation methods. Lall (2016) replicates a large number of empirical political science studies using multiple imputation instead of listwise deletion for missing values and finds that this changes the results for almost half of the studies. Unless there is a clear explanation for missingness that points to an assigned value or method, replication can test the robustness of the original results to alternative missing data techniques.…”

Influence of Missing Value Imputations on the Performance of Canonical Correspondence Analysis: Ecological Applications

“…This method although common in ecology, produces downward-biased standard errors since the zeros are treated as knowns rather than probabilistic estimates (Lall , 2016). This means that essentially this does not solve any problem in terms of quality statistical results.…”

What Drives State-Sponsored Violence?: Evidence from Extreme Bounds Analysis and Ensemble Learning Models

“…Goodness-of-fit indicators are not equivalent to the probability of a given model being true (Anscombe 1973;King 1986), and the weights constructed this way are not invariant to transformations in the dependent variable. Moreover, our data set has a number of missing observations, so model comparison measures could be misleading (Lall 2016). …”

“…Secondly, in DRF, non-observed cases are not assumed to be missing at random, but rather as values that contain information in themselves. The algorithm assumes that observations are missing for a reason, what is most likely the case with social science data (Lall 2016). This is a more conservative approach than assuming that missing cases fit into an underlying parametric distribution.…”