Towards A Differential Privacy and Utility Preserving Machine Learning Classifier

Abstract

Many organizations transact in large amounts of data often containing personal identifiable information (PII) and various confidential data. Such organizations are bound by state, federal, and international laws to ensure that the confidentiality of both individuals and sensitive data is not compromised. However, during the privacy preserving process, the utility of such datasets diminishes even while confidentiality is achieved--a problem that has been defined as NP-Hard. In this paper, we investigate a differential privacy machine learning ensemble classifier approach that seeks to preserve data privacy while maintaining an acceptable level of utility. The first step of the methodology applies a strong data privacy granting technique on a dataset using differential privacy. The resulting perturbed data is then passed through a machine learning ensemble classifier, which aims to reduce the classification error, or, equivalently, to increase utility. Then, the association between increasing the number of weak decision tree learners and data utility, which informs us as to whether the ensemble machine learner would classify more correctly is examined. As results, we found that a combined adjustment of the privacy granting noise parameters and an increase in the number of weak learners in the ensemble machine might lead to a lower classification error.

Research on an Ensemble Classification Algorithm Based on Differential Privacy

“…This method does not need to carry out discrete preprocessing of data. Literature [20], Mivule et al proposed a framework that uses AdaBoost iterations to update data sets until the forest achieves an acceptable level of prediction accuracy. However, the framework lacks detail in that it does not give the specific content of the differential privacy technology, or how it allocates the privacy budget.…”

Privacy preserving classification over differentially private data

“…Therefore, in this study, we focused on using input perturbation to perform differentially private classification task. To reach our goal, we adopted input perturbation technique of differential privacy as used in the studies of (Mivule et al, 2012; Sánchez et al, 2016; Sarwate & Chaudhuri, 2013; Senekane, 2019) to perform privacy preserving classification. We experimentally analyzed the performances of the well‐known classification algorithms that are C4.5, Naïve Bayes, Bayesian Networks, IBk, K*, One Rule, PART, Random tree, and Ripper for classification of the differentially private data, obtained by applying input perturbation to 8 widely used UCI datasets for various privacy levels by changing the ɛ values from 1 to 5, and also for small ɛ values (i.e., ɛ <1).…”

“…Input perturbation : In this technique, the data is perturbed by adding noise to the values of its numerical attributes for a certain privacy level (i.e., ɛ ).Definition Let x be a d ‐dimensional vector of a data instance in a database D , differentially private version of x can be given as in Equation (5). where z is a d ‐dimensional vector with a Laplace density probability function given in Equation ). With this noise addition to each individual data vector x i in the database D , it can be guaranteed that the resulting database D priv = ( x priv 1 , x priv 2 , x priv 3 , …, x privn ) is an ɛ ‐differentially private approximation to D (Antonova, 2016; Dwork et al, 2006; Mivule et al, 2012; Sarwate & Chaudhuri, 2013; Senekane, 2019).…”

“…It is the easiest way of providing differential privacy, and the significant advantage of the input perturbation is that, it enables to release the noisy dataset while maintaining the privacy. In other words, input perturbation is independent of any data mining algorithm unlike objective perturbation and output perturbation techniques, and researchers can utilize this private data to run their own functions on it (Antonova, 2016; Ji et al, 2014; Mivule et al, 2012; Sarwate & Chaudhuri, 2013).…”

A differential privacy protecting K-means clustering algorithm based on contour coefficients

“…Many research have been done by scholars from home and abroad. Documentary [ 4 ] proposed the issue of balance between availability and privacy of differential privacy protection. As differential privacy protection is a data distortion technique, the balance between availability and privacy of differential privacy protection is an NP problem.…”