Privacy-preserving impact evaluation using Difference-in-Differences

Abstract

Difference-in-Differences is a widely used tool for causal impact evaluation. However, its application to data containing sensitive personal information, such as individual-level health care data, is limited due to strict data privacy regulations. Obtaining consent from subjects often leads to smaller sample sizes or no individuals from treated/untreated groups, reducing statistical power or making impact estimation impossible. Federated Learning, a distributed learning approach that ensures privacy by sharing aggregated statistics, can address data protection concerns. However, software packages for advanced Difference-in-Differences algorithms are yet unavailable. We have developed a federated version of the Difference-in-Differences with multiple time periods, which we have implemented. Our federated package ensures data privacy and is able to reproduce outputs of the non-federated implementation. By leveraging federated estimates, we are able to increase effective sample sizes, reduce estimation uncertainty, and make estimation possible in cases where data owners only have access to the treated or untreated group.