Sensitivity analysis in differentially private machine learning using hybrid automatic differentiation

Abstract

In recent years, formal methods of privacy protection such as differential privacy (DP), capable of deployment to data-driven tasks such as machine learning (ML), have emerged. Reconciling large-scale ML with the closed-form reasoning required for the principled analysis of individual privacy loss requires the introduction of new tools for automatic sensitivity analysis and for tracking an individual's data and their features through the flow of computation. For this purpose, we introduce a novel hybrid automatic differentiation (AD) system which combines the efficiency of reverse-mode AD with an ability to obtain a closed-form expression for any given quantity in the computational graph. This enables modelling the sensitivity of arbitrary differentiable function compositions, such as the training of neural networks on private data. We demonstrate our approach by analysing the individual DP guarantees of statistical database queries. Moreover, we investigate the application of our technique to the training of DP neural networks. Our approach can enable the principled reasoning about privacy loss in the setting of data processing, and further the development of automatic sensitivity analysis and privacy budgeting systems.

Partial sensitivity analysis in differential privacy

“…Partial sensitivity therefore represents the fractional contribution of the individual input attributes to the gradient norm of the function. Its symbolic representation, which can be easily obtained using a symbolic AD system [4], is independent of the actual input data and can therefore be used to interpret the impact of individual input attributes on the gradient norm, used for (individual) privacy accounting, as shown in the next section.…”

“…Details on the attached code Symbolic automatic differentiation was performed using the Deuterium framework [4], whose source code is available alongside the experiment code. Compilation performed by Deuterium relies on a suitable compiler for the C or Fortran programming language or for the LLVM tool-chain.…”

“…This is, in part, because in existent machine learning frameworks, it is impossible to disentangle the interaction between input features and model components such as weights. To address this issue, recent work has introduced symbolic automatic differentiation (AD)-based sensitivity analysis [4]. The main contribution of our work is the introduction of partial sensitivity, representing the fractional contribution of each input feature to the gradient norm, which determines privacy loss in DP, and can be derived using symbolic AD.…”