Differential Privacy Principal Component Analysis for Support Vector Machines

Abstract

In big data era, massive and high-dimensional data is produced at all times, increasing the difficulty of analyzing and protecting data. In this paper, in order to realize dimensionality reduction and privacy protection of data, principal component analysis (PCA) and differential privacy (DP) are combined to handle these data. Moreover, support vector machine (SVM) is used to measure the availability of processed data in our paper. Specifically, we introduced differential privacy mechanisms at different stages of the algorithm PCA-SVM and obtained the algorithms DPPCA-SVM and PCADP-SVM. Both algorithms satisfy ε , 0 -DP while achieving fast classification. In addition, we evaluate the performance of two algorithms in terms of noise expectation and classification accuracy from the perspective of theoretical proof and experimental verification. To verify the performance of DPPCA-SVM, we also compare our DPPCA-SVM with other algorithms. Results show that DPPCA-SVM provides excellent utility for different data sets despite guaranteeing stricter privacy.

Differentially Private Singular Value Decomposition for Training Support Vector Machines

“…The three algorithms were compared theoretically between DPSVD, AG [ 9 ], and DPPCA-SVM [ 26 ] summarized in Table 2 . Other ones have been compared by the DPPCA-SVM algorithm.…”

“…Imtiaz and Sarwate [ 22 , 23 ] and Jiang et al [ 24 ] disturbed the matrix of covariance with Wishart noise and it guarantees that the perturbed matrix of covariance is positive semidefinite. Xu et al [ 25 ] and Huang et al [ 26 ] added symmetric Laplace noise to the matrix of covariance. Those methods above all generate the perturbed matrix of covariance by adding a noise matrix and then perform EVD to implement PCA.…”

“…Those methods above all generate the perturbed matrix of covariance by adding a noise matrix and then perform EVD to implement PCA. And only [ 26 ] measured the availability of the private PCA by SVM, but not to research private PCA from the privacy perspective of SVM. Recently, SVD has been widely used in collaborative filtering [ 27 ], deep learning [ 28 ], data compression [ 29 , 30 ], and images watermarking [ 31 ].…”

Differential Privacy Protection for Support Vector Machines for Nonlinear Classification

“…In DP, the sensitivity Δf can be understood as the approximate adjacency degree of the two data sets for the function f Definition 3 (Laplace Mechanism [27]). Given a data set D, for a function f: D ⟶ R d with sensitivity Δf, the mechanism M provides (ε, 0)− DP satisfying:]…”