Asynchronous privacy-preserving iterative computation on peer-to-peer networks

Security and Communication Networks

Berta

Hegedűs

et al. 2018

Self Cite

Privacy and security are among the highest priorities in data mining approaches over data collected from mobile devices. Fully distributed machine learning is a promising direction in this context. However, it is a hard problem to design protocols that are efficient yet provide sufficient levels of privacy and security. In fully distributed environments, secure multiparty computation (MPC) is often applied to solve these problems. However, in our dynamic and unreliable application domain, known MPC algorithms are not scalable or not robust enough. We propose a light-weight protocol to quickly and securely compute the sum query over a subset of participants assuming a semihonest adversary. During the computation the participants learn no individual values. We apply this protocol to efficiently calculate the sum of gradients as part of a fully distributed minibatch stochastic gradient descent algorithm. The protocol achieves scalability and robustness by exploiting the fact that in this application domain a "quick and dirty" sum computation is acceptable. We utilize the Paillier homomorphic cryptosystem as part of our solution combined with extreme lossy gradient compression to make the cost of the cryptographic algorithms affordable. We demonstrate both theoretically and experimentally, based on churn statistics from a real smartphone trace, that the protocol is indeed practically viable.

Section: Related Workmentioning

confidence: 99%

Robust Fully Distributed Minibatch Gradient Descent with Privacy Preservation

Security and Communication Networks

Berta

Hegedűs

et al. 2018

Self Cite

“…Unfortunately, this model of computation does not cover our problem as we want to compute minibatches of a size independent of the size of the direct neighborhood, and the proposed approach does not scale well in that sense. Besides, the robustness of the method is not satisfactory either [17]. Han et al address stochastic gradient search explicitly [12].…”

Section: Related Workmentioning

confidence: 99%

Fully Distributed Privacy Preserving Mini-batch Gradient Descent Learning

Lecture Notes in Computer Science

Jelasity

2015

Self Cite

Abstract. In fully distributed machine learning, privacy and security are important issues. These issues are often dealt with using secure multiparty computation (MPC). However, in our application domain, known MPC algorithms are not scalable or not robust enough. We propose a light-weight protocol to quickly and securely compute the sum of the inputs of a subset of participants assuming a semi-honest adversary. During the computation the participants learn no individual values. We apply this protocol to efficiently calculate the sum of gradients as part of a fully distributed mini-batch stochastic gradient descent algorithm. The protocol achieves scalability and robustness by exploiting the fact that in this application domain a "quick and dirty" sum computation is acceptable. In other words, speed and robustness takes precedence over precision. We analyze the protocol theoretically as well as experimentally based on churn statistics from a real smartphone trace. We derive a sufficient condition for preventing the leakage of an individual value, and we demonstrate the feasibility of the overhead of the protocol.

“…Unfortunately, this model of computation does not cover our problem as we want to compute mini-batches of a size independent of the size of the direct neighborhood, and the proposed approach does not scale well in that sense. Besides, the robustness of the method is not satisfactory either [61]. Han et al address stochastic gradient search explicitly [37].…”

Section: Related Workmentioning

confidence: 99%

Efficient Gossip Algorithms for Machine Learning

First of all, I would like to thank my supervisor, Dr. Márk Jelasity, for guiding and supporting my research and providing the opportunity to work on fascinating topics.Next, I would like to thank my colleagues who helped me to discover interesting areas of science and helped give birth to new ideas during our discussions. In alphabetical order, they are Dr. Árpád Berta, Dr. István Hegedűs and Dr. Róbert Ormándi. I would also like to thank David P. Curley for correcting this thesis from a linguistic point of view.Last but not least, I would like to thank my family and friends for supporting me all this time.