Quantization of Distributed Data for Learning

Hanna, Osama A.; Ezzeldin, Yahya H.; Fragouli, Christina; Diggavi, Suhas

doi:10.1109/jsait.2021.3105359

Cited by 18 publications

(8 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This process achieved up to 4.86 times improvement in the network lifetime [ 33 ]. They also modified an algorithm wherein the sink can predict its next movement and presented simulation results to verify this approach [ 34 ].…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Optimization of Leach Protocol in Wireless Sensor Network Using Machine Learning

Ramesh

Rajasekaran

Dwivedi

et al. 2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

The Wireless Sensor Network is a network formed in areas human beings cannot access. The data need to be sensed by the sensor and transferred to the sink node. Many routing protocols are designed to route data from a single node to the sink node. One of the routing protocols is the hierarchical routing protocol, which passes on the sensed data hierarchically. The Low Energy Adaptive Clustering Hierarchy (LEACH) is one of the hierarchical methods in which communication happens in two steps: the setup phase and the steady-state phase. The efficiency of the LEACH has to be optimized to improve the network lifetime. Therefore, the k-means clustering algorithm, which comes under the unsupervised machine learning method, is incorporated with the LEACH algorithm and has shown better results. But the selection of cluster head needs to improvise because it will transfer the summed-up data to the sink node, so it is to be efficient enough. So, this paper proposes the modified k-means algorithm with LEACH protocol for optimizing the Wireless Sensor Network. In the modified k-means algorithm, the weight of the cluster head is tested and elected, and the clusters are formed using the Euclidean distance formula. The proposed work yields 48.85% efficiency compared to the existing protocol. It is also proven that the proposed work showed more successful data transfer to the sink node. The cluster head selection process elects the more efficient node as the head with less failure rate. The proposed work optimistically balanced the whole network in terms of energy and successful data transfer.

show abstract

Section: Resultsmentioning

confidence: 99%

“…Step 11: Repeat from step 3 until all the nodes are dead. also modified an algorithm wherein the sink can predict its next movement and presented simulation results to verify this approach [34]. Data delivery latency influenced the speed of the mobile sink.…”

Section: Resultsmentioning

confidence: 99%

Optimization of Leach Protocol in Wireless Sensor Network Using Machine Learning

Ramesh

Rajasekaran

Dwivedi

et al. 2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

show abstract

“…Note that studying "how quantization of covariate affects the recovery/learning" is meaningful especially when the problem is interpreted as sparse linear regression -working with low-precision data in some (distributed) learning systems could significantly reduce communication cost and power consumption [43], [96], which we will further demonstrate in Section IV-A. However, almost all of the prior works are restricted to response quantization.…”

Section: A Related Workmentioning

confidence: 99%

“…Indeed, to reduce the power consumption and computational cost, it is sometimes preferable to work with low-precision data in a machine learning system, e.g., the sample quantization scheme developed in [96] led to experimental success in training linear model. Also, it was shown that direct gradient quantization may not be efficient in certain distributed learning systems where the terminal nodes are connected to the server only through a very weak communication fabric and the number of parameters is extremely huge; rather, quantizing and transmitting some important samples could provably reduce communication cost [43]. In fact, the process of data collection may already appeal to quantization due to certain limits of the data acquisition device (e.g., a low-resolution analog-to-digital module used in distributed signal processing [25]).…”

Section: A Covariate Quantizationmentioning

confidence: 99%

“…In some applications, relatively low resolution would be preferable. For instance, in a distributed learning setting or a MIMO system, the frequent information transmission among multiple parties often results in prohibitive communication cost [56], [69], and quantizing signals or data to fairly low resolution is an effective approach to reduce the cost [43], [96]. Under such a big picture, in recent years there has been rapidly growing literature on high-dimensional signal recovery from quantized data (see, e.g., [7], [22], [26], [32], [33], [50], [89] for 1-bit quantization, [46], [50], [89], [94] for multi-bit uniform quantization), trying to understand the interplay between quantization and signal reconstruction in some fundamental estimation problems.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Quantizing Heavy-Tailed Data in Statistical Estimation: (Near) Minimax Rates, Covariate Quantization, and Uniform Recovery

Chen,

Ng,

Wang

2024

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

Modern datasets often exhibit heavy-tailed behavior, while quantization is inevitable in digital signal processing and many machine learning problems. This paper studies the quantization of heavy-tailed data in several fundamental statistical estimation problems where the underlying distributions have bounded moments of some order (no greater than 4). We propose to truncate and properly dither the data prior to a uniform quantization. Our major standpoint is that (near) minimax rates of estimation error could be achieved by computationally tractable estimators based on the quantized data produced by the proposed scheme. In particular, concrete results are worked out for covariance estimation, compressed sensing (also interpreted as sparse linear regression), and matrix completion, all agreeing that the quantization only slightly worsens the multiplicative factor. Additionally, while prior results focused on the quantization of responses (i.e., measurements), we study compressed sensing where the covariates (i.e., sensing vectors) are also quantized; in this case, though our recovery program is non-convex (since our covariance matrix estimator lacks positive semi-definiteness), we prove that all local minimizers enjoy near-optimal estimation error. Moreover, by the concentration inequality of the product process and a covering argument, we establish a near minimax uniform recovery guarantee for quantized compressed sensing with heavy-tailed noise. Finally, numerical simulations are provided to corroborate our theoretical results.

show abstract