Multitask Diffusion Adaptation Over Networks With Common Latent Representations

Abstract

Online learning with streaming data in a distributed and collaborative manner can be useful in a wide range of applications. This topic has been receiving considerable attention in recent years with emphasis on both single-task and multitask scenarios. In single-task adaptation, agents cooperate to track an objective of common interest, while in multitask adaptation agents track multiple objectives simultaneously. Regularization is one useful technique to promote and exploit similarity among tasks in the latter scenario. This work examines an alternative way to model relations among tasks by assuming that they all share a common latent feature representation. As a result, a new multitask learning formulation is presented and algorithms are developed for its solution in a distributed online manner. We present a unified framework to analyze the mean-square-error performance of the adaptive strategies, and conduct simulations to illustrate the theoretical findings and potential applications.

Performance Analysis of Diffusion LMS for Cyclostationary White Non-Gaussian Inputs

“…Behavior model (21) implies that the mean weight error behavior of DLMS depend on the type of periodic input variance σ 2 x,k (n) and its period T instead of the random perturbation of optimal weight vector and the type of white non-Gaussian input distribution u k (n) [12].…”

“…The associate editor coordinating the review of this manuscript and approving it for publication was Xiao-Sheng Si. extended to multitask scenarios [19]- [21]. Nevertheless, the most of existing analytical models of DLMS are limited to under the stationary Gaussian assumption on the input distributions [19], [22]- [24].…”

The Algorithms of Distributed Learning and Distributed Estimation about Intelligent Wireless Sensor Network

“…With the development of automation and wireless communication technology, data acquisition, data processing, data transmission, data control and data storage are not only convenient and fast, but also safe and reliable in a complicated network [1,2]. Combining distributed computing, computer science, automatic control theory, wireless sensor, and microelectronics manufacturing, intelligent network is a kind of large-scale distributed network systems, which is the integration of data-aware, intelligent learning, dynamic optimization, and wireless data communication [3,4].…”

Online Proximal Learning Over Jointly Sparse Multitask Networks With $\ell _{\infty, 1}$ Regularization

“…For the latter, a subgradient and a proximal algorithm are introduced in [13] and [14], respectively. In [15] and [16], the authors derive solutions for other classes of multitask problems where the relations between the nodes are defined by common latent representations or local linear equality constraints, respectively. In [17], the authors solve a multitask problem by estimating the combination matrix.…”