Distributed Lossy Averaging

Su, Han-I; Gamal, Abbas El

doi:10.1109/tit.2010.2048474

Cited by 10 publications

(18 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…terms are neglected, in the limit of zero-distortion (high resolution) 7 . Similar with the data-aggregation case, this solution for network consensus is also very similar to the The classical outer bound [11,Prop. 4] about the distributed network consensus in a tree network is O nlog 2 1 n 3/2 D .…”

Section: A Inner and Outer Bounds Based On Incremental-distortionmentioning

confidence: 69%

“…In each time slot, we assume that only one node transmits along only one edge. We follow the notion of distributed computation scheme introduced in [11]. By a distributed computation scheme, we denote a five-tuple (T, S , G , v, e) described in the following.…”

Section: System Model and Problem Formulationmentioning

confidence: 99%

“…In both cases, the difference between the inner and outer bounds is shown to approach zero in the high-resolution (i.e., zero distortion) limit.Note that in [10, Section V], the authors show a constant difference between their lower and the 2 Note that the original bound in [11] has a normalization term 1 n . The bound in [11] is useful only if D = O(n − 3 2 ). Our outer bound is useful for all D. 5 reconstruction or function computation.…”

mentioning

confidence: 99%

“…However, the above-mentioned references do not consider noiseless lossy in-network computation.Some previous works on information-theoretic distributed computing also rely on randomcoding-based techniques to provide inner bounds [10], [26]. The achievable schemes in [11] utilize Gaussian test channels, which also implicitly require random coding arguments. However, we find it hard to directly analyze the random coding schemes for distributed lossy computing with Gaussian sources, especially for computing the overall mean-square error of the consensus value, because we may need to obtain a non-trivial generalization of the "Markov Lemma" [32, Lecture Notes 13] to Gaussian sources (see Remark 5 for details).…”

mentioning

confidence: 99%

See 3 more Smart Citations

Rate Distortion for Lossy In-Network Linear Function Computation and Consensus: Distortion Accumulation and Sequential Reverse Water-Filling

Yang

Grover

Kar

2017

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

We consider the problem of distributed lossy linear function computation in a tree network. We examine two cases: (i) data aggregation (only one sink node computes) and (ii) consensus (all nodes compute the same function). By quantifying the accumulation of information loss in distributed computing, we obtain fundamental limits on network computation rate as a function of incremental distortions (and hence incremental loss of information) along the edges of the network. The above characterization, based on quantifying distortion accumulation, offers an improvement over classical cut-set type techniques which are based on overall distortions instead of incremental distortions. This quantification of information loss qualitatively resembles information dissipation in cascaded channels [1]. Surprisingly, this accumulation effect of distortion happens even at infinite blocklength. Combining this observation with an inequality on the dominance of mean-square quantities over relative-entropy quantities, we obtain outer bounds on the rate distortion function that are tighter than classical cut-set bounds by a difference which can be arbitrarily large in both data aggregation and consensus. We also obtain inner bounds on the optimal rate using random Gaussian coding, which differ from the outer bounds by, where D is the overall distortion. The obtained inner and outer bounds can provide insights A preliminary version of this work was presented in part at the 53rd Annual Allerton Conference on Communication, Control and Computing, 2015.2 on rate (bit) allocations for both the data aggregation problem and the consensus problem. We show that for tree networks, the rate allocation results have a mathematical structure similar to classical reverse water-filling for parallel Gaussian sources. I. INTRODUCTIONThe phenomenon of information dissipation [1]- [6] has been of increasing interest recently from an information-theoretic viewpoint. These results characterize and quantify the gradual loss of information as it is transmitted through cascaded noisy channels. This study has also yielded data processing inequalities that are stronger than those used classically [2], [5].The dissipation of information cannot be quantified easily using classical information-theoretic tools that rely on the law of large numbers, because the dissipation of information is often due to finite-length of codewords and power constraints on the channel inputs [1]. In many classical network information theory problems, such as relay networks, the dissipation of information is not observed because it can be suppressed by use of asymptotically infinite blocklengths [1],[7] 1 . However, information dissipation does happen in many problems of communications and computation. For example, in [4], Evans and Schulman obtain bounds on the information dissipation in noisy circuits, and in [1], Polyanskiy and Wu examine a similar problem in cascaded AWGN channels with power-constrained inputs. Our earlier works [8], [9] show that under some conditions, error-corre...

show abstract

Section: A Inner and Outer Bounds Based On Incremental-distortionmentioning

confidence: 69%

Section: System Model and Problem Formulationmentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Rate Distortion for Lossy In-Network Linear Function Computation and Consensus: Distortion Accumulation and Sequential Reverse Water-Filling

Yang

Grover

Kar

2017

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

show abstract

“…A handful of works (e.g., [23][24][25][26]) analyze consensus from the viewpoint of information theory. Yang et al [26] considered RD bounds for data aggregation, in which data is routed through a tree network to a fusion center, and consensus, in which each node forms an estimate of the desired quantity.…”

Section: A Prior Artmentioning

confidence: 99%

Generalized geometric programming for rate allocation in consensus

Pilgrim

Zhu

Baron

et al. 2017

2017 55th Annual Allerton Conference on Communication, Control, and Computing (Allerton)

View full text Add to dashboard Cite

Distributed averaging, or distributed average consensus, is a common method for computing the sample mean of the data dispersed among the nodes of a network in a decentralized manner. By iteratively exchanging messages with neighbors, the nodes of the network can converge to an agreement on the sample mean of their initial states. In real-world scenarios, these messages are subject to bandwidth and power constraints, which motivates the design of a lossy compression strategy. Few prior works consider the rate allocation problem from the perspective of constrained optimization, which provides a principled method for the design of lossy compression schemes, allows for the relaxation of certain assumptions, and offers performance guarantees. We show for Gaussian-distributed initial states with entropy-coded scalar quantization and vector quantization that the coding rates for distributed averaging can be optimized through generalized geometric programming. In the absence of side information from past states, this approach finds a rate allocation over nodes and iterations that minimizes the aggregate coding rate required to achieve a target mean square error within a finite run time. Our rate allocation is compared to some of the prior art through numerical simulations. The results motivate the incorporation of sideinformation through differential or predictive coding to improve rate-distortion performance.

show abstract