Privacy-preserving linear and nonlinear approximation via linear programming

Abstract

We propose a novel privacy-preserving random kernel approximation based on a data matrix A ∈ R m×n whose rows are divided into privately owned blocks. Each block of rows belongs to a different entity that is unwilling to share its rows or make them public. We wish to obtain an accurate function approximation for a given y ∈ R m corresponding to each of the m rows of A. Our approximation of y is a real function on R n evaluated at each row of A and is based on the concept of a reduced kernel K(A, B ′ ) where B ′ is the transpose of a completely random matrix B. The proposed linear-programming-based approximation, which is public but does not reveal the privately-held data matrix A, has accuracy comparable to that of an ordinary kernel approximation based on a publicly disclosed data matrix A.Keywords: privacy-preserving approximation, random kernels, support vector machines, linear programming INTRODUCTIONThe problem addressed in this work is that of obtaining an approximation to a given vector y ∈ R m of function values corresponding to the m rows of a data matrix A ∈ R m×n that represents m points in the n-dimensional real space R n . The matrix A is partitioned into q blocks of rows belonging to q entities that are unwilling to share their data or make them public. The motivation for this work arises from similar problems arising in classification theory where the data, corresponding to rows of a data matrix, is also held by various private entities and hence referred to as horizontally partitioned data. Thus in [19,15] privacy-preserving support vector machine (SVM) classifiers were obtained for such data, while in [20] induction tree classifiers were generated for similar problems. Other privacypreserving classifying techniques include cryptographically private SVMs [7], wavelet-based distortion [10] and rotation perturbation [3]. There is also a substantial body of research on privacy preservation in linear programming such as [1,12,13]. However, there does not appear to be any privacy-preserving applications to approximation problems in the literature. This is the problem we wish to address here as follows.In this work we propose an efficient privacy-preserving approximation (PPA) for horizontally partitioned data that is based on the following two ideas. For a given data matrix A ∈ R m×n , instead of using the usual kernel function K(A, A ′ ) : R m×n × R n×m −→ R m×m for constructing a linear or nonlinear approximation of a given y ∈ R m corresponding to the m rows of A, we use a random kernel [9, 8] K(A, B ′ ) : R m×n × R n×m −→ R m×m ,m < n, where B is a completely random matrix that is publicly disclosed. Such a random kernel will be shown to completely hide the data matrix A. The second idea is that each entity i ∈ {1, . . . , q} makes public only the kernel function K(A i , B ′ ) of its

Privacy Preservation in Distributed Subgradient Optimization Algorithms

“…The above equation implies that z 1 = a 11 , z 2 = a 12 due to x 1 (0) = x 2 (0). Similarly, we can show z 3 = a 21 , z 4 = a 22 . This implies that the matrix equations in Theorem 3.1 has a unique solution, which yields a contradiction.…”

“…In cryptograph-based methods [27], agents need to encrypt the estimates needed to be shared with their neighbors and decrypt the received estimates so that agents' privacy cannot be disclosed. In differential privacy methods [17,19,21,22,26], agents need to add random noises on the estimates needed to transmitted to protect agents' privacy. Different from them, in our algorithm (9), the estimates can be transmitted directly between neighbors without any additional technique to disguise agents' estimates.…”

“…The existing privacy preserving methods can be roughly classified into two classes: cryptograph-based approaches [27] and non-cryptograph-based approaches [17,19,21,22,26]. In cryptograph-based methods, many mechanisms are designed to encrypt the data needed to be transmitted and decrypt the received data so that the privacy is not disclosed.…”

Big Data for weed control and crop protection

“…There have been significant advances in privacy‐aware data analytics (Dreier & Kerschbaum, ; Fung & Mangasarian, ; Sarwate & Chaudhuri, ; Yan et al ., ; Duchi et al ., ; Weeraddana et al ., ; Xie et al ., ; Nozari et al ., ). However, a significant amount of research and development is still needed to devise tools which simultaneously ensure the following: 1) very high degrees of privacy, 2) handling huge data volumes and 3) handling very generic machine learning models or tasks.…”

“…Naturally, such transformations are sought to be ‘non‐invertible’, in the sense that the adversary cannot (or is very unlikely) to recover private data by observing the messages. For example, Dreier and Kerschbaum () and Fung and Mangasarian () address solving LPs, where each party multiplies its own real‐valued private data vector D i by a privately generated random matrix . The main advantage of the non‐cryptography‐based solutions with respect to the cryptography‐based ones is that data analytics is performed directly over the transformed, but non‐encrypted data (real numbers, vectors and matrices).…”