Distributed Inference Over Directed Networks: Performance Limits and Optimal Design

Abstract

We find large deviations rates for consensus-based distributed inference for directed networks. When the topology is deterministic, we establish the large deviations principle and find exactly the corresponding rate function, equal at all nodes. We show that the dependence of the rate function on the stochastic weight matrix associated with the network is fully captured by its left eigenvector corresponding to the unit eigenvalue. Further, when the sensors' observations are Gaussian, the rate function admits a closed form expression. Motivated by these observations, we formulate the optimal network design problem of finding the left eigenvector which achieves the highest value of the rate function, for a given target accuracy. This eigenvector therefore minimizes the time that the inference algorithm needs to reach the desired accuracy. For Gaussian observations, we show that the network design problem can be formulated as a semidefinite (convex) program, and hence can be solved efficiently. When observations are identically distributed across agents, the system exhibits an interesting property: the graph of the rate function always lies between the graphs of the rate function of an isolated node and the rate function of a fusion center that has access to all observations. We prove that this fundamental property holds even when the topology and the associated system matrices change randomly over time, with arbitrary distribution. Due to generality of its assumptions, the latter result requires more subtle techniques than the standard large deviations tools, contributing to the general theory of large deviations.

Optimal Aggregation Strategies for Social Learning over Graphs

“…This is because, in the adaptive social learning problem, the agents want to learn an optimal decision from the received data, rather than a specific parameter. From (24), the performance of learning is determined by the distribution of x ave,i (θ), which captures the information of local observations across the network. When the signal model is accurate, the log-likelihood ratio x k,i (θ) provides the full information of each piece of independent observation ξ k,i for the decision-making task.…”

“…It was shown in [21] that the diffusion-based detection strategy achieves a better adaptation ability than the consensus-based counterpart. The learning performance of both strategies, i.e., the learning speed and the steady-state error probability were shown to be dependent on the combination policy employed by the network [19]- [22], [24].…”

Communication efficient distributed weighted non-linear least squares estimation

“…For future reference, we also introduce the mean Laplacian matrix {L(t)} as L(t) = E [L(t)], and L(t) = L(t) − L(t). Thus, it holds that E L(t) = 0, and E L(t) 2…”

“…1 From now on, in order to better distinguish the MSE rate of decay with respect to the number of iterations t and with respect to the number of per-node communications, we will refer to the former as the MSE iteration-wise rate and to the latter as the MSE communication rate. 2 The stronger requirement imposed here, with 1 being strictly positive, is only required for the benchmark estimator in Eqs. (13)- (14) ahead to be defined properly; the reason for this requirement is the two time scale nature [43], and the red line represents the proposed estimator of the benchmark estimator (13)- (14).…”

“…The agents acquire their local, noisy, non-linear observations about the unknown phenomenon (unknown static vector parameter θ) in a streaming fashion over discrete time instances t. The goal for each agent is to continuously generate an estimate of θ over time instances t in a recursive fashion, where the estimate update of an agent involves simultaneous assimilation of the newly acquired local observations, and the received information through messages with agents in its immediate neighborhood. The assumed setup is highly relevant in several emerging applications in the context of cyber-physical systems (CPS) and the Internet of things (IoT), like state *Correspondence: djakovet@uns.ac.rs 2 University of Novi Sad, Faculty of Sciences, Department of Mathematics and Informatics, 21000 Novi Sad, Serbia Full list of author information is available at the end of the article estimation in smart grid, predictive maintenance, and production monitoring in industrial manufacturing systems. For example, with continuous state estimation of a smart grid, the acquired measurements (voltages, angles) are in general non-linear functions of the unknown state; further, the measurements are inherently distributed across different physical locations (elements of the system), and they arrive continuously over time with a prescribed sampling rate.…”