Detecting concept drift in fully distributed environments

Hegedűs, István; Nyers, Lehel; Ormándi, Róbert

doi:10.1109/sisy.2012.6339511

Cited by 7 publications

(7 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The method needs to dynamically account for new browsing history and incorporate this into the collaborative filtering process. One method based on gossip learning that introduces dynamism into the model is that of concept drift [13]. The drift detection rests on an adaptive mechanism that uses the historical performances of the models circulating in the network.…”

Section: Algorithm 1 Svd Updatementioning

confidence: 99%

Decentralized Collaborative Filtering for Distributed Hash Tables

Shukla

2024

Preprint

View full text Add to dashboard Cite

We implement a decentralized method for collaborative filtering via matrix factorization that can rest on peer-to-peer Distributed Hash Table architectures like Pastry and Chord, for the purpose of realizing a scalable and robust decentralised search engine. We begin by introducing a centralized method for matrix factorization that involves minimizing a cost function consisting of the ratings and user and item latent factors. We introduce biases and then derive the gradients for the error update step. We then introduce gossip learning; a matrix-factorization based collaborative filtering approach for a decentralized network. Since we could not obtain a dataset of browsing activity of users where the rating would constitute the number of times a website is visited, we used a dataset of movie ratings to test our method. This preserves the user and item biases and provides user-item ratings for rating to latent factor error computation. We performed a train-test split and loaded the training and validation set onto each node separately. This method is then tested using the PeerSim peer-to-peer network simulator to obtain the best learning rate $\eta$, regularization parameter $\lambda$ and latent factors $k$ for a $10,000$ node peer-to-peer network. The optimization metric used is the Root Mean Squared Error measuring the discrepancy between the ratings in the validation set and their respective predictions by model updates on the training set at each node. We found that by the fortieth cycle of the PeerSim simulator, the training set is fully distributed across each node. The best performing parameter setting over our $63$ experiment runs has learning rate $\eta = 0.001$, regularization parameter $\lambda = 1$ and latent factors $k = 20$. These settings are then built into a basic working prototype of a Pastry algorithm based peer-to-peer distributed search engine with message passing tested for $50$ nodes.

show abstract

Section: Algorithm 1 Svd Updatementioning

confidence: 99%

Decentralized Collaborative Filtering for Distributed Hash Tables

Shukla

2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Other methods such as Gossip-based learning framework [5] detect and handle concept drift in peer-to-peer (P2P) networks. This algorithm does not collect data at a central location, instead it uses a handful of online learners taking random walks in the network [6] and training on each node's dataset at each step.…”

Section: Related Workmentioning

confidence: 99%

Attentive Federated Learning for Concept Drift in Distributed 5G Edge Networks

Estiri¹,

Maheswaran²

2021

Preprint

View full text Add to dashboard Cite

Machine learning (ML) is expected to play a major role in 5G edge computing. Various studies have demonstrated that ML is highly suitable for optimizing edge computing systems as rapid mobility and application-induced changes occur at the edge. For ML to provide the best solutions, it is important to continually train the ML models to include the changing scenarios. The sudden changes in data distributions caused by changing scenarios (e.g., 5G base station failures) is referred to as concept drift and is a major challenge to continual learning. The ML models can present high error rates while the drifts take place and the errors decrease only after the model learns the distributions. This problem is more pronounced in a distributed setting where multiple ML models are being used for different heterogeneous datasets and the final model needs to capture all concept drifts. In this paper, we show that using Attention in Federated Learning (FL) is an efficient way of handling concept drifts. We use a 5G network traffic dataset to simulate concept drift and test various scenarios. The results indicate that Attention can significantly improve the concept drift handling capability of FL.

show abstract

“…This is very hard to do even without any extra measures, given that models perform random walks based on local decisions, and that merge operations are performed as well. This short informal reasoning motivates our ongoing work towards understanding and enhancing the privacy-preserving properties of gossip learning, while we extend the basic idea to tackle with different learning scenarios (like concept drift [53,55]), and different models (like boosting models [54], multi-armed bandit models [114]). The presented results are mainly based on our previous paper [94].…”

Section: Discussionmentioning

confidence: 99%

“…Chapter 3 Chapter 4 Chapter 5 Chapter 6 Chapter 7 ICDM 2008 [89] • TSD 2010 [91] • EUROPAR 2010 [92] • WETICE 2010 [90] • EUROPAR 2011 [93] • • CCPE 2012 [94] • SASO 2012 [55] • • SISY 2012 [53] • • EUROPAR 2012 [54] • • ICML 2013 [114] • • from the instantiations.…”

Section: Introductionmentioning

confidence: 99%

Applications of Support Vector-Based Learning

Ormándi¹

View full text Add to dashboard Cite

PrefaceAn increasing amount of data is stored in electronic form. This phenomenon increases the need of the methods that help to access the useful pieces of information that are hidden in ocean of the collected data. That is, methods to perform automated data processing are needed. The role of machine learning is to provide methods that support this progress, i.e. it provides algorithms that can automatically discover those hidden and non-trivial patterns in data that are useful for us.In the focus of this thesis stands a special family of machine learning algorithms referred to as support vector-based learners. These algorithms are all based on the so-called maximal margin heuristic, which helps to select the final model from the suitable ones preserving the generalization ability of the model. These methods are quite general, and they have a numerous amazing properties like generalization capability or robustness to noisy data. However, to apply them to a specific task it is needed to adapt them to that particular task. This is challenging since the inappropriate adaptation can result in a drastic decrease in prediction performance or in the generalization capability; or cause a computationally infeasible situation (e.g. huge computational complexity or untreatable network load in a distributed setting).The goal of this thesis is to examine the suitability and adaptability of ii various support vector-based learning methods in numerous learning tasks and environments. During this, novel techniques and ideas, which are less frequent or maybe surprising, are investigated and discussed. The considered tasks themselves touch a wide range problems like time series forecasting, opinion mining, collaborative filtering, and the common binary classification problem. An uncommon computational environment is also concerned, namely the fully distributed environment.Róbert Ormándi, July 2013.

show abstract

Detecting concept drift in fully distributed environments

Cited by 7 publications

References 28 publications

Decentralized Collaborative Filtering for Distributed Hash Tables

Decentralized Collaborative Filtering for Distributed Hash Tables

Attentive Federated Learning for Concept Drift in Distributed 5G Edge Networks

Applications of Support Vector-Based Learning

Contact Info

Product

Resources

About