Proving differential privacy with shadow execution

Abstract

Recent work on formal verification of differential privacy shows a trend toward usability and expressiveness -generating a correctness proof of sophisticated algorithm while minimizing the annotation burden on programmers. Sometimes, combining those two requires substantial changes to program logics: one recent paper is able to verify Report Noisy Max automatically, but it involves a complex verification system using customized program logics and verifiers.In this paper, we propose a new proof technique, called shadow execution, and embed it into a language called Shad-owDP. ShadowDP uses shadow execution to generate proofs of differential privacy with very few programmer annotations and without relying on customized logics and verifiers. In addition to verifying Report Noisy Max, we show that it can verify a new variant of Sparse Vector that reports the gap between some noisy query answers and the noisy threshold. Moreover, ShadowDP reduces the complexity of verification: for all of the algorithms we have evaluated, type checking and verification in total takes at most 3 seconds, while prior work takes minutes on the same algorithms.CCS Concepts • Software and its engineering → Formal software verification.• [i] 1 i := 0; bq := 0; max := 0; 2 while ( i < size ) 3 η := Lap (2/ϵ) , Ω ? † : • , Ω ? 2 : 0 ; 4 if ( q [ i ] + η > bq ∨ i = 0) 5 max := i; 6 bq := q[i] + η; 7 i := i + 1; The transformed program (slightly simplified for readability), where underlined commands are added by the type system: 1 v ϵ := 0; 2 i := 0; bq := 0; max := 0; 3 bq • := 0; bq † := 0; 4 while ( i < size ) 5 assert (i < size); 6 havoc η; v ϵ := Ω ? (0 + ϵ) : (v ϵ + 0); 7 if ( q [ i ] + η > bq ∨ i = 0) 8 assert (q[i] + q • [i] + η + 2 > bq + bq † ∨ i = 0); 9 max := i; 10 bq † := bq + bq † -(q[i] + η); 11 bq := q[i] + η; 12 bq • := q • [i] + 2; 13 else 14 assert (¬(q[i] + q • [i] + η + 0 > bq + bq • ∨ i = 0)); 15 // shadow execution 16 if (q[i] + q † [i] + η > bq + bq † ∨ i = 0) 17 bq † := q[i] + q † [i] + η − bq; 18 i := i + 1;

A Programming Language for Data Privacy with Accuracy Estimations

“…Related work sensitivity-and its derivatives DFuzz [21], Adaptive Fuzz [58], Fuzzi [62], and Duet [44]. Hoare2 [4], a programming language which enforces (pure or approximate) di erential privacy using program veri cation, together with its extension PrivInfer [5] supporting di erentially private Bayesian programming; and other systems using similar ideas [7,1,61,57].…”

Asymmetric Differential Privacy

“…A DP mechanism can be so complicated (e.g., the sparse vector technique described in the following section) that the wrong proofs appear [25]. Then, many verification tools using a proof syntax [2,32,34] are proposed to verify the correctness of proof for DP. The idea of the proof syntax is called randomness alignment.…”

A Programming Framework for Differential Privacy with Accuracy Concentration Bounds

“…Within this group are Fuzz [4]-a programming language which enforces (pure) differential privacy of computations using a linear type system which keeps track of program sensitivity-and its derivatives DFuzz [6], Adaptive Fuzz [10], Fuzzi [13], and Duet [50]. Hoare2 [7], a programming language which enforces (pure or approximate) differential privacy using program verification, together with its extension PrivInfer [8] supporting differentially private Bayesian programming; and other systems using similar ideas [43,51,9,52].…”