Best practices for addressing missing data through multiple imputation

Abstract

A common challenge in developmental research is the amount of incomplete and missing data that occurs from respondents failing to complete tasks or questionnaires, as well as from disengaging from the study (i.e., attrition). This missingness can lead to biases in parameter estimates and, hence, in the interpretation of findings. These biases can be addressed through statistical techniques that adjust for missing data, such as multiple imputation. Although multiple imputation is highly effective, it has not been widely adopted by developmental scientists given barriers such as lack of training or misconceptions about imputationThe foundation for this paper was created during a 'hackathon' session occurring on 23 June 2021, at the annual virtual meeting of the Society for Improving Psychological Science. We invited anyone interested in the topic to attend, welcoming both experts and those with little experience addressing missing data in their research, specifically welcoming participation from those who were not sure how to address the missing data they experienced.Decisional guidelines for analyzing the type and extent of missing data were then crowdsourced and curated during this hackathon, resulting in a missing data and multiple imputation decision tree (Woods et al., 2021, available at https://doi.org/10.31234/osf.io/mdw5r) and a companion infographic (Woods & Schmidt, 2021, available at https://miro.com/app/board/o9_J18JGJQk=/). We also created multiple imputation coding templates for several prominent software languages (Stata, Mplus, R, SPSS, SAS and Blimp). All hackathon materials and coding templates are available at https://osf.io/j3f8m/.

Charting the longitudinal trajectories and interplay of critical consciousness among youth activists

“…Second, the attrition rate for this sample, particularly at time 3 where it was 79%, is another limitation of the current study. Presumably, this attrition stemmed from a more geographically mobile sample that has demonstrated greater attrition in previous longitudinal research (Woods et al, 2021). Further, some attrition was caused by the nature of YO—a data collection was canceled because a YO group mobilized to protest the Muslim Ban at the local airport, for example.…”

“…We included a set of auxiliary variables (reviewed above) in the imputation phase of the analysis to make the MAR assumption more tenable and improve the efficiency of the imputation model (Enders, 2010). However, although we include gender and race as auxiliary variables in our imputation model, we do not impute missing values for gender and race as it does not seem appropriate to impute identity markers, which may misrepresent participants' true experiences (Woods et al, 2021). A total of 500 imputed datasets were generated because when missingness is more severe, the fraction of missing information–which quantifies the degree to which missingness affects a parameter's sampling error while accounting for all the information in the covariance matrix–tends to stabilize at 500 imputations (Madley‐Dowd et al, 2019).…”

Examining Intergenerational Risk Factors for Conduct Problems Using Polygenic Scores in the Norwegian Mother, Father and Child Cohort Study

“…As data was likely not missing completely at random due to previously reported selective attrition (58), we imputed missing data by multiple imputation using chained equations as implemented in the R package mice v3.13.0 (59), and followed recent guidance (60), creating 100 imputed data sets with 30 iterations to achieve convergence, and pooling results from the imputed data sets. Details on the imputation process, including additional variables used for imputation and results using imputed data can be found in the supplementary note and in Tables S3 and S6.…”

Open science and metascience in developmental psychology: Introduction to the special issue

“…Turoman et al (2022) present a workflow for applying open science principles in a developmental psychology lab, using their own lab as an example. Regarding data analysis, Visser et al (2023) present a tutorial for using Bayesian sequential testing designs and Woods et al (2023) present best practices for addressing missing data through multiple imputations.…”