When Machine Learning Meets Blockchain: A Decentralized, Privacy-preserving and Secure Design

Learning to Learn Gradient Aggregation by Gradient Descent

“…The idea of learning to learn was inspired by psychology studies [Ward, 1937], where researchers discovered that the human beings are born to be able to learn new skills via previous learning experiences. The idea of learning to learn has been widely explored in various machine learning systems such as meta-learning, life-long learning and continuous learning, as well as many optimization problems, including learning to optimize hyperparameters [Maclaurin et al, 2015], learning to optimize neural networks [Ge et al, 2017] and learning to optimize loss functions [Houthooft et al, 2018].…”

“…We notice that this procedure can easily map onto a recurrent neural network (RNN), which learns to utilize its internal memory to store information about previous processes and function evaluations, and learns to access this memory to produce current output. To address the gradients vanishing or exploding issues when dealing with long sequential data, in this paper, we apply two popular and efficient variants of the standard RNN, RNN with Long Short-Term Memory units (LSTM) [Hochreiter and Schmidhuber, 1997] and RNN with Gated Recurrent units (GRU) [Chung et al, 2014] .…”

“…With the onset of the big data era, governments, companies and research institutions for the first time have the opportunity to gain deep insight into their problems and develop better solutions by leveraging massive available data [Dean et al, 2012;. However, developing machine learning algorithms to analyze a huge amount of data usually requires intensive computing resources, which may not be provided by any single user.…”

“…Moreover, the massive data are typically generated by multiple parties and stored * Contact Author in a geographically distributed manner. Such scenarios with distributed computing power and datasets have spurred the study of distributed machine learning techniques, such as multi-agent optimization [Nedic and Ozdaglar, 2009], multiparty learning [Hamm et al, 2016], and federated learning [Konečnỳ et al, 2016].…”

“…The most typical distributed machine learning paradigm is the master-slave pattern, and has been adopted in many industrial-level applications [Abadi et al, 2016;Chen et al, 2018b]. Particularly, each worker (the slave) in the system is a computing node, which has access to its own share of the data.…”