Privacy Preservation in Distributed Subgradient Optimization Algorithms

Abstract

Privacy preservation is becoming an increasingly important issue in data mining and machine learning. In this paper, we consider the privacy preserving features of distributed subgradient optimization algorithms. We first show that a well-known distributed subgradient synchronous optimization algorithm, in which all agents make their optimization updates simultaneously at all times, is not privacy preserving in the sense that the malicious agent can learn other agents' subgradients asymptotically. Then we propose a distributed subgradient projection asynchronous optimization algorithm without relying on any existing privacy preservation technique, where agents can exchange data between neighbors directly. In contrast to synchronous algorithms, in the new asynchronous algorithm agents make their optimization updates asynchronously. The introduced projection operation and asynchronous optimization mechanism can guarantee that the proposed asynchronous optimization algorithm is privacy preserving. Moreover, we also establish the optimal convergence of the newly proposed algorithm. The proposed privacy preservation techniques shed light on developing other privacy preserving distributed optimization algorithms.

Privacy-Preserving Incremental ADMM for Decentralized Consensus Optimization

“…Inspired by the privacy-preservation definitions in [33], [47], [48], we define the privacy as follows.…”

Algorithm-Level Confidentiality for Average Consensus on Time-Varying Directed Graphs

“…This results in the disclosure of sensitive state information, which is sometimes undesirable in terms of confidentiality. In fact, in many applications such as the smart grid, health-care or banking networks, confidentiality is crucial for promoting participation in collaboration since individual agents tend not to trade confidentiality for performance [9]- [11]. For instance, a group of people using average consensus to reach a common opinion may want to keep their individual opinions secret [12].…”

ADMM Based Privacy-Preserving Decentralized Optimization

“…To enable privacy in decentralized optimization without incurring large communication/computational overhead or compromising algorithmic accuracy, we propose a novel privacy solution through function decomposition. In the optimization literature, privacy has been defined as preserving the confidentiality of agents' states [22], (sub)gradients or objective functions [20], [30], [31]. In this paper, we define privacy as the non-disclosure of agents' (sub)gradients.…”