Private Inner Product Retrieval for Distributed Machine Learning

Abstract

In this paper, we argue that in many basic algorithms for machine learning, including support vector machine (SVM) for classification, principal component analysis (PCA) for dimensionality reduction, and regression for dependency estimation, we need the inner products of the data samples, rather than the data samples themselves.Motivated by the above observation, we introduce the problem of private inner product retrieval for distributed machine learning, where we have a system including a database of some files, duplicated across some non-colluding servers. A user intends to retrieve a subset of specific size of the inner products of the data files with minimum communication load, without revealing any information about the identity of the requested subset. For achievability, we use the algorithms for multi-message private information retrieval. For converse, we establish that as the length of the files becomes large, the set of all inner products converges to independent random variables with uniform distribution, and derive the rate of convergence. To prove that, we construct special dependencies among sequences of the sets of all inner products with different length, which forms a time-homogeneous irreducible Markov chain, without affecting the marginal distribution. We show that this Markov chain has a uniform distribution as its unique stationary distribution, with rate of convergence dominated by the second largest eigenvalue of the transition probability matrix. This allows us to develop a converse, which converges to a tight bound in some cases, as the size of the files becomes large. While this converse is based on the one in multi-message private information retrieval due to the nature of retrieving inner products instead of data itself some changes are made to reach the desired result.

Single-Server Private Linear Transformation: The Individual Privacy Case

“…1) Construction of the matrix G: The user constructs an L(n + m) × K matrix G with a structure similar to (10), where G 1 , . .…”

“…. , 8 of V correspond respectively to the message indices in W, i.e., 5,8,11,2,4,7,10,18. Thus, the user constructs the permutation π such that π(5) = 9, π(8) = 10, π(11) = 11, π(2) = 12, π(4) = 13, π(7) = 14, π (10) = 15, π(18) = 16.…”

“…, 24}. Note that the columns 10,11,12,16,17,18,22,23,24 of the matrix G are constructed based on the columns 1, . .…”

“…The IPLT problem is also related to the problem of singleserver PLT with joint privacy guarantees (or JPLT for short), which we have studied in a parallel work [6]. The notion of joint privacy was previously considered for the problems of multi-message PIR [7]- [10] and single-server PLC [5], [11]. The joint privacy condition implies that, from the server's perspective, every D-subset of messages must be equally likely a posteriori to be the support set of the L required linear combinations.…”

“…Since H generates a [10,5] MDS code, it can also be thought of as the parity-check matrix of a [10,5] MDS code. The user then takes G 3 to be the generator matrix of the [10,5] MDS code defined by the parity-check matrix H, 3 given by ( 22)-( 24), the user constructs the matrix G as in (21).…”