Decentralized Online Learning: Take Benefits from Others' Data without Sharing Your Own to Track Global Trend

Abstract

Decentralized Online Learning (online learning in decentralized networks) attracts more and more attention, since it is believed that Decentralized Online Learning can help the data providers cooperatively better solve their online problems without sharing their private data to a third party or other providers. Typically, the cooperation is achieved by letting the data providers exchange their models between neighbors, e.g., recommendation model. However, the best regret bound for a decentralized online learning algorithm is O n √ T , where n is the number of nodes (or users) and T is the number of iterations. This is clearly insignificant since this bound can be achieved without any communication in the networks. This reminds us to ask a fundamental question: Can people really get benefit from the decentralized online learning by exchanging information? In this paper, we studied when and why the communication can help the decentralized online learning to reduce the regret. Specifically, each loss function is characterized by two components: the adversarial component and the stochastic component. Under this characterization, we show that decentralized online gradient (DOG) enjoys a regret bound O √ n 2 T G 2 + nT σ 2 , where G measures the magnitude of the adversarial component in the private data (or equivalently the local loss function) and σ measures the randomness within the private data. This regret suggests that people can get benefits from the randomness in the private data by exchanging private information. Another important contribution of this paper is to consider the dynamic regret -a more practical regret to track users' interest dynamics. Empirical studies are also conducted to validate our analysis.

Cooperative Online Learning: Keeping your Neighbors Updated

“…The communication model includes delays, and the regret bound depends on a quantity related to the mixing time of a certain random walk on the network. Zhao et al [18] study a decentralized online learning setting in which losses are characterized by two components, one adversarial and another stochastic. They show upper bounds on the regret in terms of a constant representing the magnitude of the adversarial component and another constant measuring the randomness of the stochastic part.…”

Central Server Free Federated Learning over Single-sided Trust Social Networks

“…Examples of these works include Kamp et al (2014); Shahrampour and Jadbabaie (2017); Lee et al (2016). Notably, Zhao et al (2019) shares similar problem definition and theoretical result as our paper. However, single sided communication is not allowed in their setting, making their results more restrictive.…”

“…t for any i and t, then Algorithm 1 reduces to the distributed online gradient method proposed by Zhao et al (2019).…”

Accelerating Distributed Online Meta-Learning via Multi-Agent Collaboration under Limited Communication

“…However, little attention has been paid to understand the impact of the network size on the average regret achievable at each individual agent therein. To reap the potential benefits that an agent can achieve when carrying out distributed OCO, a convex loss function with both adversarial and stochastic components is considered in [36]. Assuming that the expected gradient is bounded above by 𝐺 and the stochastic variance is bounded above by 𝜎 2 , they have shown that the network expected regret is 𝑂 ( √ 𝑁 2 𝐺 2 𝑇 + 𝑁𝑇 𝜎 2 ).…”

“…Remark 2. The classical DOGD algorithm [36] cannot achieve such performance gain in the setting here, because essentially DOGD performs a consensus step followed by a gradient descent along the local gradient ∇𝑓 𝑡,𝑖 (𝑥 𝑡,𝑖 ). For a fixed learning rate, DOGD only converges to a neighborhood of the optimizer 𝑥 * , because the local gradient is data-driven and hence random.…”