Finding a Flexible Hot-Deck Imputation Method for Multinomial Data

Abstract

Detailed breakdowns on totals are often collected in surveys, such as a breakdown of total product sales by product type. These multinomial data are often sparsely reported with wide variability in proportions across units. In addition, there are often true zeros that differ across units even within industry; for example, one establishment sells jeans but not shoes, and another sells shoes but not socks. It is quite common to have large fractions of missing data for these detailed items, even when totals are relatively completely observed. Hot-deck imputation, which fills in missing data with observed data values, is an attractive approach. The entire set of proportions can be simultaneously imputed to preserve multinomial distributions, and zero values can be imputed. However, it is not clear what variant of the hot deck is best. We describe a large set of “flavors” of the hot deck and compare them through simulation and by application to data from the 2012 Economic Census. We consider different ways to create the donor pool: choosing one nearest neighbor (NN), choosing from five NNs, or using all units as the donor pool. We also consider different ways to impute from the donor: directly impute the donor’s vector of proportions or randomly draw from a multinomial distribution using this vector of proportions. We consider scenarios where a strong predictor of these multinomial distributions exists as well as when covariate information is weak.

Nearest neighbor ratio imputation with incomplete multi-nomial outcome in survey sampling

“…Second, if the proportion of recipients to donors is large, then NNRI may repeatedly use the same donor, yielding insufficient variation within each imputation cell. Andridge et al (2021) propose a modification of the NNRI method that addresses this issue in a multiple imputation framework; it would be useful to develop a single imputation analogue. Third, in practice, many auxiliary variables can be used to determine nearest neighbors, in which case, dimension reduction is necessary to mitigate matching discrepancies.…”

“…The simulation varies in four factors: parametric distribution of the size (auxiliary) variable x i , the size of the finite population (N ), relationship of auxiliary variable and detail items (x i and y i ), and response propensity. The data generation is largely patterned after the realistic procedures described in Andridge, Bechtel, and Thompson (2021), with each separate process outlined below. The first population scenario ensures the compact and convex support requirements of Assumption 3.…”

“…The variance estimation is further complicated in our setting due to the potential shift in multinomial distributions as described above. Andridge, Bechtel, and Thompson (2021) investigates multiple imputations of proportions with nearest neighbor ratio hot deck imputation using the Approximate Bayesian Bootstrap, finding consistent underestimation of the variance. As in the cited reference, we assume that the set of details is reported or is missing; in practice, detail components that do not sum to their associated total and are not within a small raking tolerance are often treated as missing, and all detail items are imputed, regardless of their original reporting status.…”

“…The empirical application highlights the differences between our proposed variance estimator from a naive variance estimator but does not provide insight into its statistical properties. Consequently, Section 5 investigates the finite-sample performance of the proposed over repeated stratified SRS-WOR samples via a simulation study patterned off of Andridge et al (2021). We close in Section 6 with some general observations along with directions for future research.…”

Nearest Neighbour Ratio Imputation with Incomplete Multinomial Outcome in Survey Sampling

“…The variance estimation is further complicated in our setting due to the potential shift in multinomial distributions as described above. Andridge et al (2021) investigates multiple imputations of proportions with nearest neighbour ratio hot deck imputation using the Approximate Bayesian Bootstrap, finding consistent underestimation of the variance. As in the cited reference, we assume that the set of details is reported or is missing; in practice, detail components that do not sum to their associated total and are not within a small raking tolerance are often treated as missing, and all detail items are imputed, regardless of their original reporting status.…”