Differentially Private Collaborative Coupling Learning for Recommender Systems

Abstract

Coupling learning is designed to estimate, discover and extract the interactions and relationships among learning components. It provides insights into complex interactive data, and has been extensively incorporated into recommender systems to enhance the interpretability of sophisticated relationships between users and items. Coupling learning can be further fostered once the trending collaborative learning can be engaged to take advantage of the cross-platform data. To facilitate this, privacy-preserving solutions are in high demand-it is desired that the collaboration should not expose either the private data of each individual owner or the model parameters trained on their datasets. In this work, we develop a distributed collaborative coupling learning system which enables differential privacy. The proposed system defends against the adversary who has gained full knowledge of the training mechanism and the access to the model trained collaboratively. It also addresses the privacy-utility tradeoff by a provable tight sensitivity bound. Our experiments demonstrate that the proposed system guarantees favourable privacy gains at a modest cost in recommendation quality, even in scenarios with a large number of training epochs. Index Terms-Coupling learning, differential privacy, collaborative learning COUPLING LEARNING is an emerging research topic that refers to understanding, formalizing and quantifying the complex relations and interactions, i.e., couplings hidden in complex data. Effective discovery and extraction of the

Confined Gradient Descent: Privacy-preserving Optimization for Federated Learning

“…Another line of studies that approaches to privacy-preserving FL is through differential privacy (DP) mechanism [2,8,12,16,21,42,44,48,49]. The common practice of achieving differential privacy is based on additive noise calibrated to ∇𝐹 's sensitivity S 2 ∇𝐹 .…”

“…In the traditional FL, every participant owns its local training dataset, and updates the same global model 𝑤 𝑘 via a parameter server using its local model/gradients. The local gradients ∇𝐹 (𝑤 𝑘 , 𝜉 𝑙,𝑘 ) can be protected via either secure aggregation [4,30,31,34,40,47] or differential privacy mechanisms [2,8,21,42,44,48]. This process can be decentralized by replacing the parameter server with a peer-to-peer communication mechanism [22,39].…”

“…The sharing of one and the same global model (even though the local gradients are protected) is still subject to information leakage (refer to [35] and Section 5). Privacy-preserving SGD via differential privacy [2,8,21,42,44,48] Additive noise (in most cases) affects the model accuracy [8].…”

“…Each of them obtains the current model parameters, works out a local gradient based on the local data, and disseminates it to update the global model synchronously [2,4] or asynchronously [21]. This paradigm guarantees data locality, but has been found insufficient for data privacy: although the local gradients can be securely synthesized via a variety of techniques such as differential privacy (DP) [2,21,42,48], secure multi-party communication (MPC) [4,11,34], and homomorphic encryption (HE) [31,34,40], the global model parameters that have to be shared are still susceptible to information leakage(cf. Section 5 and [35,36]).…”

PrivColl: Practical Privacy-Preserving Collaborative Machine Learning

“…There also have been some previous research efforts which have explored collaborative learning without exposing their trained models [24,33,46,47]. For example, Papernot et al [33] make use of transfer learning in combination with differential privacy to learn an ensemble of teacher models on data partitions, and then use these models to train a private student model.…”