Calibrated Hot-Deck Donor Imputation Subject to Edit Restrictions

Abstract

A major challenge faced by basically all institutes that collect statistical data on persons, households or enterprises is that data may be missing in the observed data sets. The most common solution for handling missing data is imputation. Imputation is complicated owing to the existence of constraints in the form of edit restrictions that have to be satisfied by the data. Examples of such edit restrictions are that someone who is less than 16 years old cannot be married in the Netherlands, and that someone whose marital status is unmarried cannot be the spouse of the head of household. Records that do not satisfy these edits are inconsistent, and are hence considered incorrect. A further complication when imputing categorical data is that the frequencies of certain categories are sometimes known from other sources or have previously been estimated. In this article we develop imputation methods for imputing missing values in categorical data that take both the edit restrictions and known frequencies into account.

Obtaining numerically consistent estimates from a mix of administrative data and surveys

“…As RI is implemented by applying one or more imputation methods that preserve edit rules and previously estimated population totals, we select one such imputation method for the comparison. The imputation method we select is the calibrated hot deck imputation method proposed by [4] for categorical data. Actually, in [4] several calibrated hot deck imputation methods, depending on how hot deck is actually carried out, are described.…”

“…The imputation method we select is the calibrated hot deck imputation method proposed by [4] for categorical data. Actually, in [4] several calibrated hot deck imputation methods, depending on how hot deck is actually carried out, are described. For the purposes of the current paper, the differences between these methods are not important, and will we discuss them as if they are the same.…”

Calibrated Hot Deck Imputation for Numerical Data Under Edit Restrictions

“…This approach is based on solving a mathematical optimization problem that can be exceedingly large. Beaumont (2005) Favre, Matei and Tillé (2005) and Coutinho, De Waal and Shlomo (2013) have developed methods for categorical data having to satisfy edits and to preserve totals. An obvious difference between those methods and our approach is we focus on numerical data.…”

Statistical Disclosure Limitation in the Presence of Edit Rules

“…This involves an error localization step, for example, using the methods of Fellegi and Holt (1976), followed by replacing the localized errors with imputations that respect constraints. For example, one could use sequential regression imputation (Van Buuren and Oudshoorn 1999;Raghunathan et al 2001), imputation from joint distributions (Geweke 1991;Tempelman 2007;Coutinho et al 2011;Kim et al 2014b), or in some settings hot-deck imputation (Bankier et al 1994;Shlomo and De Waal 2005;Coutinho and De Waal 2012;Coutinho et al 2013). As examples of this strategy, Shlomo and De Waal (2008) apply several SDL methods and correct edit-failing records via an edit-imputation procedure based on linear programming; and Cano and Torra (2011) propose adding random noise followed by swapping the noise values of edit-failing records until all records pass edit constraints.…”