Communication efficient distributed weighted non-linear least squares estimation

Abstract

The paper addresses design and analysis of communication-efficient distributed algorithms for solving weighted non-linear least squares problems in multi-agent networks. Communication efficiency is highly relevant in modern applications like cyber-physical systems and the Internet of things, where a significant portion of the involved devices have energy constraints in terms of limited battery power. Furthermore, non-linear models arise frequently in such systems, e.g., with power grid state estimation. In this paper, we develop and analyze a non-linear communication-efficient distributed algorithm dubbed CRE DO − N L (non-linear CRE DO). CRE DO − N L generalizes the recently proposed linear method CRE DO (Communication efficient REcursive Distributed estimatOr) to non-linear models. We establish for a broad class of non-linear least squares problems and generic underlying multi-agent network topologies CRE DO − N L's strong consistency. Furthermore, we demonstrate communication efficiency of the method, both theoretically and by simulation examples. For the former, we rigorously prove that CRE DO − N L achieves significantly faster mean squared error rates in terms of the elapsed communication cost over existing alternatives. For the latter, the considered simulation experiments show communication savings by at least an order of magnitude.

Non-Asymptotic Rates for Communication Efficient Distributed Zeroth Order Strongly Convex Optimization

“…In contrast to [14][15][16], we consider a communication efficient distributed zeroth optimization scheme, where we explicitly characterize the communication savings while ensuring order-optimal convergence rates as compared to [17]. In prior work [18,19], we developed distributed algorithms with increasingly sparse communications for statistical estimation problems. This paper demonstrates that the concept of increasingly sparse communications can be exploited to develop communication-efficient distributed zeroth order stochastic optimization algorithms also.…”

“…This paper demonstrates that the concept of increasingly sparse communications can be exploited to develop communication-efficient distributed zeroth order stochastic optimization algorithms also. Technically, the setups in [18,19] and the setup here are very different, requiring new analyses. Communication efficient distributed estimation schemes as proposed in [18,19] involve local correctness, i.e., the optimizers of the sum of loss functions of the individual nodes is a subset of the optimizers of each local function, while in the current work, the setup is rendered locally incorrect.…”

“…Technically, the setups in [18,19] and the setup here are very different, requiring new analyses. Communication efficient distributed estimation schemes as proposed in [18,19] involve local correctness, i.e., the optimizers of the sum of loss functions of the individual nodes is a subset of the optimizers of each local function, while in the current work, the setup is rendered locally incorrect. We skip the proofs due to space limitations.…”

Communication-Efficient Distributed Strongly Convex Stochastic Optimization: Non-Asymptotic Rates

“…Finally, we review the class of works that are concerned with designing distributed methods that achieve communication efficiency, e.g., [2], [22]- [27]. In [26], a data censoring method is employed in the context of distributed least squares estimation to reduce computational and communication costs.…”

“…These strategies are different from ours; we utilize randomized, increasingly sparse communications in general. In references [2], [27], we study distributed estimation problems and develop communication-efficient distributed estimators. The problems studied in [2], [27] have a major difference with respect to the current paper in that, in [2], [27], the assumed setting yields individual nodes' local gradients to evaluate to zero at the global solution.…”