Distributed adaptive online learning for convex optimization with weight decay

Abstract

This paper investigates an adaptive gradient‐based online convex optimization problem over decentralized networks. The nodes of a network aim to track the minimizer of a global time‐varying convex function, and the communication pattern among nodes is captured as a connected undirected graph. To tackle such optimization problems in a collaborative and distributed manner, a weight decay distributed adaptive online gradient algorithm, called WDDAOG, is firstly proposed, which incorporates distributed optimization methods with adaptive strategies. Then, our theoretical analysis clearly illustrates the difference between weight decay and L2 regularization for distributed adaptive gradient algorithms. The dynamic regret bound for the proposed algorithm is further analyzed. It is shown that the dynamic regret bound for convex functions grows with order of scriptOfalse(nfalse(1+logTfalse)+nTfalse), where T and n represent the time horizon and the number of nodes associated with the network, respectively. Numerical experiments demonstrate that WDDAOG works well in practice and compares favorably to existing distributed online optimization schemes.

Distributed online adaptive subgradient optimization with dynamic bound of learning rate over time‐varying networks

“…Thus, we need to solve the following optimization problem $$\begin{equation} \begin{aligned} \underset{{\bf x} \in \mathcal {F}}{\operatornamewithlimits{minimize}}\quad F({\bf x})= \sum _{t=1}^{T} \sum _{i=1}^{n} f_{i, t}({\bf x}), \end{aligned} \end{equation}$$where $$f_{i, t}: \mathcal {F} \rightarrow \mathbb {R}$$ is a time‐varying convex and potentially nonsmooth objective function. To embody the online essence of the problem (4), we select the dynamic regret [8, 29, 30] as the metric, in which the cumulative loss of all agents is compared against the best sequence $$\lbrace {\bf x}_{t}^{*}\rbrace _{t=1}^{T}$$, that is $$\begin{equation} R(T)= \sum _{i=1}^{n} \sum _{t=1}^{T} f_{i, t}{\left({\bf x}_{i, t}\right)}-\sum _{t=1}^{T} f_{t}{\left({\bf x}_{t}^{*}\right)}, \end{equation}$$where $${\bf x}_{t}^{*}=\arg \min _{{\bf x} \in \mathcal {F}} f_{t}({\bf x})$$. If the dynamic regret of an online algorithm is sublinear, that is, …”

“…where f i,t ∶  → ℝ is a time-varying convex and potentially nonsmooth objective function. To embody the online essence of the problem (4), we select the dynamic regret [8,29,30] as the metric, in which the cumulative loss of all agents is compared against the best sequence {x * t } T t =1 , that is…”

An accelerated distributed online gradient push‐sum algorithm on time‐varying directed networks

“…If the benchmark is a sequence of time-varying optimal solutions, the regret is called dynamic regret. For instance, the authors in Shen et al [12] proposed a weight decay distributed adaptive online gradient algorithm proving a dynamic regret bound of …”

“…In the regret analysis, we prove the individual regret to be sublinear by establishing relationship between the network regret and the individual regret. In particular, substituting (13) into (12), the difference between R 𝑗 (t) and R (T) can be written as a cumulative sum of network disagreement terms and reflects the consensus errors among agents.…”

“…If the benchmark is a sequence of time‐varying optimal solutions, the regret is called dynamic regret. For instance, the authors in Shen et al [12] proposed a weight decay distributed adaptive online gradient algorithm proving a dynamic regret bound of . The work in Zhao et al [13] obtained a decentralized online gradient descent method and was shown to achieve a regret bound of for the convex case.…”