Peer-to-Peer Multi-class Boosting

Abstract

Abstract. We focus on the problem of data mining over large-scale fully distributed databases, where each node stores only one data record. We assume that a data record is never allowed to leave the node it is stored at. Possible motivations for this assumption include privacy or a lack of a centralized infrastructure. To tackle this problem, earlier we proposed the generic gossip learning framework (GoLF), but so far we have studied only basic linear algorithms. In this paper we implement the well-known boosting technique in GoLF. Boosting techniques have attracted growing attention in machine learning due to their outstanding performance in many practical applications. Here, we present an implementation of a boosting algorithm that is based on FilterBoost. Our main algorithmic contribution is a derivation of a pure online multi-class version of FilterBoost, so that it can be employed in GoLF. We also propose improvements to GoLF, with the aim of maximizing the diversity of the evolving models gossiped in the network, a feature that we show to be important. We evaluate the robustness and the convergence speed of the algorithm empirically over three benchmark databases. We compare the algorithm with the sequential AdaBoost algorithm and we test its performance in a failure scenario involving message drop and delay, and node churn.

Applications of Support Vector-Based Learning

“…This is very hard to do even without any extra measures, given that models perform random walks based on local decisions, and that merge operations are performed as well. This short informal reasoning motivates our ongoing work towards understanding and enhancing the privacy-preserving properties of gossip learning, while we extend the basic idea to tackle with different learning scenarios (like concept drift [53,55]), and different models (like boosting models [54], multi-armed bandit models [114]). The presented results are mainly based on our previous paper [94].…”

“…Chapter 3 Chapter 4 Chapter 5 Chapter 6 Chapter 7 ICDM 2008 [89] • TSD 2010 [91] • EUROPAR 2010 [92] • WETICE 2010 [90] • EUROPAR 2011 [93] • • CCPE 2012 [94] • SASO 2012 [55] • • SISY 2012 [53] • • EUROPAR 2012 [54] • • ICML 2013 [114] • • from the instantiations.…”

Gossip-Based Machine Learning in Fully Distributed Environments

“…Since the nodes in the network store locally the received models (as its CURRENTMODEL or the latest models can be collected in a bounded queue), they can use them to predict the labels of new instances without additional commu- wait(∆) 4: p ← selectPeer() 5: send currentModel to p 6: end loop currentModel ← m 10: end procedure nication cost. Moreover, incoming models can be combined as well, both locally (e.g., merging the received models, or implementing a local voting mechanism on the models in the queue) [88,89] or globally (e.g., finding the best model in the network) [52].…”

“…In this chapter we present a well-known technique in GOLF, called boosting, which was published in our paper [52]. Boosting techniques have attracted growing attention in machine learning due to their outstanding performance in many practical applications.…”

“…Chapter 3 Chapter 4 Chapter 5 Chapter 6 CCPE 2013 [89] • • • • EUROPAR 2012 [52] • • SASO 2012 [53] • • SISY 2012 [50] • • ACS 2013 [51] • • P2P 2014 [49] • • EUROPAR 2011 [88] • ICML 2013 [106] • ESANN 2014 [18] • TIST 2016 [47] • • PDP 2016 [48] • PDP 2016 [8] • • a fully distributed way and instances of the matrix Y perform random walks in the network. When a Y visits a node, it gets updated based on the local row of A, and the local row of X gets updated as well.…”