Overlay Management for Fully Distributed User-Based Collaborative Filtering

Abstract

Abstract. Offering personalized recommendation as a service in fully distributed applications such as file-sharing, distributed search, social networking, P2P television, etc, is an increasingly important problem. In such networked environments recommender algorithms should meet the same performance and reliability requirements as in centralized services. To achieve this is a challenge because a large amount of distributed data needs to be managed, and at the same time additional constraints need to be taken into account such as balancing resource usage over the network. In this paper we focus on a common component of many fully distributed recommender systems, namely the overlay network. We point out that the overlay topologies that are typically defined by node similarity have highly unbalanced degree distributions in a wide range of available benchmark datasets: a fact that has important-but so far largely overlooked-consequences on the load balancing of overlay protocols. We propose algorithms with a favorable convergence speed and prediction accuracy that also take load balancing into account. We perform extensive simulation experiments with the proposed algorithms, and compare them with known algorithms from related work on wellknown benchmark datasets.

Nearest Neighbors Graph Construction: Peer Sampling to the Rescue

“…In such systems, each peer is connected to a subset of other peers in the network and periodically exchanges some information with one of its neighbors. While such protocols have been initially used to build uniform random topologies [19,21], they have also been applied in the context of several applications to cluster peers according to some specific metric (interest, overlap, etc) to build networks of arbitrary structure [18,24] or to support various applications such as query expansion [7], top-k queries [6] or news recommendation [9]. In such a system, the use of random nodes ensures that connectivity is maintained, each node is responsible to discover its KNN nodes by periodically exchanging neighborhood information with other peers.…”

The changing role of pervasive middleware: From discovery and orchestration to recommendation and planning

“…On the one hand, since having a single mean for properly recommending in a variety of scenarios is illusionary, such algorithms should define sort of general mechanisms flexibly and easily tunable to application-specific needs. On the other hand, given the inherent decentralization of pervasive scenarios, they should be capable of handling distributed data processing and be in themselves distributed [Orm10].…”

Applications of Support Vector-Based Learning

“…We measure prediction performance, examine its convergence and dynamics, and we measure load balancing as well. This work is mainly based on our two previous publications [90,92].…”

“…These algorithms are simple and robust, but are capable of calculating only simple functions such as the average. Nevertheless, these simple functions can serve as key components for more sophisticated methods, such as the EM algorithm [73], unsupervised learners [109], or the collaborative filtering based recommender algorithms [10,51,92,117]. Usually, these approaches use other well-studied P2P services like some kind of overlay support, for example, T-MAN [61] (for more details related to the T-MAN protocol, please see Alg.…”

“…Chapter 3 Chapter 4 Chapter 5 Chapter 6 Chapter 7 ICDM 2008 [89] • TSD 2010 [91] • EUROPAR 2010 [92] • WETICE 2010 [90] • EUROPAR 2011 [93] • • CCPE 2012 [94] • SASO 2012 [55] • • SISY 2012 [53] • • EUROPAR 2012 [54] • • ICML 2013 [114] • • from the instantiations.…”