Differential privacy in the 2020 US census: what will it do? Quantifying the accuracy/privacy tradeoff

Abstract

Background: The 2020 US Census will use a novel approach to disclosure avoidance to protect respondents’ data, called TopDown. This TopDown algorithm was applied to the 2018 end-to-end (E2E) test of the decennial census. The computer code used for this test as well as accompanying exposition has recently been released publicly by the Census Bureau. Methods: We used the available code and data to better understand the error introduced by the E2E disclosure avoidance system when Census Bureau applied it to 1940 census data and we developed an empirical measure of privacy loss to compare the error and privacy of the new approach to that of a (non-differentially private) simple-random-sampling approach to protecting privacy. Results: We found that the empirical privacy loss of TopDown is substantially smaller than the theoretical guarantee for all privacy loss budgets we examined. When run on the 1940 census data, TopDown with a privacy budget of 1.0 was similar in error and privacy loss to that of a simple random sample of 50% of the US population. When run with a privacy budget of 4.0, it was similar in error and privacy loss of a 90% sample. Conclusions: This work fits into the beginning of a discussion on how to best balance privacy and accuracy in decennial census data collection, and there is a need for continued discussion.

How differential privacy will affect our understanding of health disparities in the United States

“…Traditional disclosure techniques applied to the 2010 census counts include record-swapping, item imputation, whole household imputation, rounding, and top-and bottom-coding (52). The 2010 Demonstration Data Products, which implement DP, work by allocating a "privacy-loss budget" or e (53). The 2010 DP counts were produced under a global e = 6, where personal records use e = 4 and housing records use e = 2 (54).…”

Demonstrating an approach for evaluating synthetic geospatial and temporal epidemiologic data utility: Results from analyzing >1.8 million SARS-CoV-2 tests in the United States National COVID Cohort Collaborative (N3C)

“…Our results relate to Flaxman et al, 2020 which observed a similar effect resulting from a non-negativity constraint in the US Census' TopDown differential privacy algorithm. [12] The magnitude of the synthetic error skewing negative in a smaller concentration of zip codes increased as a key indicator became less frequent, which is fundamentally a signal problem in low-density data sets and is not specific to synthetic data generation.…”

“…Other Consortial Authors*: Christopher G. Chute 1,2,3,4,5,6,7,8,9,10 , Jon D. Morrow 1,12,13,2,7,9 , Melissa A. Haendel 14,6,10,11 1 clinical data model expertise, 2 data curation, 3 data integration, 4 data quality assurance, 5 funding acquisition, 6 governance, 7 critical revision of the manuscript for important intellectual content, 8 N3C Phenotype definition, 9 project evaluation, 10 project management, 11 regulatory oversight / admin, 12 clinical subject matter expertise, 13 data analysis, 14 funding acquisition *Consortial authorship and corresponding contributions were self-reported as part of the N3C authorship committee review process.…”

“…[3] Applying privacy-preserving methods to data comes at varying cost to utility, producing a privacy-utility trade-off. [9,[12][13][14][15] De-identification removes granular geographic information such as street-level address.…”

Testing differential privacy with dual interpreters

“…These benchmark algorithms' differential privacy proofs cover a wide range of complexity, demonstrating DPCheck can analyze both simple and sophisticated differentially private programs. (4) We present a practical workflow that uses DPCheck to re-implement and test the core differential privacy mechanisms in the Disclosure Avoidance System (DAS) [Petti and Flaxman 2019] designed for 2020 US Census; we also show statistical evidence that our re-implemented core mechanism behaves the same as the unmodified DAS (Section 7.2). (5) We implement DPCheck as an embedded language in Haskell and discuss a type-driven optimization adapted from Torlak and Bodik [2014] to speed up symbolic execution, which improves testing time for some our benchmark algorithms (Section 8).…”