A consensus-based decentralized training algorithm for deep neural networks with communication compression

Liu, Bo; Ding, Zhengtao

doi:10.1016/j.neucom.2021.01.020

Cited by 13 publications

(4 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The system must first clean the collected data, remove the features that contain default values and a large number of duplicate data, and extract various features that affect the signal coverage quality. Then, the selected features are fed into the prediction model to train the best signal coverage quality prediction model [15], [16]. Specifically, the analysis model shown in Figure 2 is mainly composed of the signal quality prediction model and the base station deployment planning model.…”

Section: Big Data Base Station Traffic Processing Model Integrating C...mentioning

confidence: 99%

Traffic Processing Model of Big Data Base Station Based on Hybrid Improved CNN Algorithm and K-Centroids Clustering Algorithm

Xu¹,

Shen³

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Wireless communication network (WCN) is very important for providing convenient mobile network communication services. Random Phase Multiple Access (RPMA) WCN under heterogeneous network architecture is widely used in wireless network construction worldwide due to its low power consumption and high cell density. However, this kind of WCN cannot meet the application scenario of high communication quality. Therefore, this research builds an RPMA communication quality prediction model for big data wireless base stations, which combines the convolutional neural network (CNN) algorithm and the lifting regression tree algorithm. It can be used to find the elements that have obvious influence on the communication quality. The reason for choosing the convolutional neural network algorithm is that its nonlinear feature relationship search and processing ability is excellent, and its computational complexity is relatively small. However, it is a black box algorithm and cannot obtain the importance coefficients of each feature, which is not conducive to subsequent analysis. Therefore, it is also necessary to select a relatively simple modified regression tree algorithm to participate in the calculation. The model constructed by integrating convolutional neural network and lifting regression tree algorithm is the RPMA wireless big data base station communication quality prediction model. A base station planning and deployment model based on weighted K-centroids algorithm is designed to obtain a better base station deployment scheme. In the CNN-DT model, the importance coefficients of T_B_diff, P_La and P_Lo are the largest and significantly larger than those of other features, which are 0.352, 0.289 and 0.264 respectively. The weighted K-centroids clustering algorithm designed in this study has the best overall downlink reception signal (RSSI) value distribution. For RSSI bucket ''-140∼-130'', the number of test points of WK centroids model, K-means model, GMC model, Mean Shift model and spectral clustering model accounted for 1.95%, 6.25%, 4.25%, 8.22% and 7.13% respectively. The model constructed in this study based on CNN and improved regression tree algorithm can accurately predict the communication quality of wireless big data base stations. This study's main contribution is that it can be used in conjunction with the base station planning and deployment model based on weighted K-centroids algorithm to improve the accuracy and effectiveness of location selection for the RPMA wireless communication network base station.INDEX TERMS CNN, promote the regression tree, K-centroids, big data base station, WCN.

show abstract

Section: Big Data Base Station Traffic Processing Model Integrating C...mentioning

confidence: 99%

Traffic Processing Model of Big Data Base Station Based on Hybrid Improved CNN Algorithm and K-Centroids Clustering Algorithm

Xu¹,

Shen³

et al. 2023

IEEE Access

View full text Add to dashboard Cite

show abstract

“…The findings show faster convergence and model correctness than standard optimizations strategies. Complex multidimensional situations may challenge this model [9]. Chen, M., et al (2021) cites.…”

Section: Literature Surveymentioning

confidence: 99%

Crow Way: An Optimization Technique for generating the Weight and Bias in Deep CNN

Papalkar,

Alvi,

Rathod

et al. 2023

ijmst

View full text Add to dashboard Cite

Fine-tuning Convolutional Neural Networks (CNNs) weights and biases are essential for solving difficult machine learning problems. We provide a hybrid optimizations strategy that combines the benefits of Crowd Search (CSA) and Brainstorm Optimization (BSO). CSA-BSO optimizes CNNs. CSA-BSO optimizes CNN weights. Our fitness measures determine each crow's location and speed throughout the CSA phase. The BSO phase mixes optimal replies with random ones to create new solutions. The best new ideas move on. After a certain number of iterations, we alternate between the CSA and BSO phases and pick the best solution as the CNN's final state. We design and test the CSA-BSO approach on many CNN architectures and datasets. Experimentally, the hybrid algorithm outperforms CSA and BSO in convergence time and solution quality. CSA and BSO collaborate to efficiently employ feasible regions and investigate a bigger search field, boosting optimizations. Stable and adaptable, CSA-BSO supports many CNN architectures and data formats. This study's main contribution is the CSA-BSO, a unique hybrid optimizations approach for convolutional neural network weight and bias optimizations. It beats CSA and BSO. The recommended strategy improves CNN picture classification, object recognition, and NLP.

show abstract

“…In this context, many algorithms have been published recently. For example in the context of targeting expensive communication problems, the authors in [5] succeeded to well-known FL algorithms, such as FedAvg and FedProx, FedSim emphasizes the significance of these contributions. In [23], the authors proposed training in heterogeneous model aggregation (MHAT) to be able to solve local model problems containing various network architectures.…”

Section: Introductionmentioning

confidence: 99%

“…Therefore, to meter the splitting process followed in this case the model will not address issues of FL related to both statistical and system heterogeneity. Expensive communication [5] MNIST [6] FedAvg variants…”

mentioning

confidence: 99%

A Heterogeneous Federated Transfer Learning Approach with Extreme Aggregation and Speed

et al. 2022

View full text Add to dashboard Cite

Federated learning (FL) is a data-privacy-preserving, decentralized process that allows local edge devices of smart infrastructures to train a collaborative model independently while keeping data localized. FL algorithms, encompassing a well-structured average of the training parameters (e.g., the weights and biases resulting from training-based stochastic gradient descent variants), are subject to many challenges, namely expensive communication, systems heterogeneity, statistical heterogeneity, and privacy concerns. In this context, our paper targets the four aforementioned challenges while focusing on reducing communication and computational costs by involving recursive least squares (RLS) training rules. Accordingly, to the best of our knowledge, this is the first time that the RLS algorithm is modified to completely accommodate non-independent and identically distributed data (non-IID) for federated transfer learning (FTL). Furthermore, this paper also introduces a newly generated dataset capable of emulating such real conditions and of making data investigation available on ordinary commercial computers with quad-core microprocessors and less need for higher computing hardware. Applications of FTL-RLS on the generated data under different levels of complexity closely related to different levels of cardinality lead to a variety of conclusions supporting its performance for future uses.

show abstract

A consensus-based decentralized training algorithm for deep neural networks with communication compression

Cited by 13 publications

References 25 publications

Traffic Processing Model of Big Data Base Station Based on Hybrid Improved CNN Algorithm and K-Centroids Clustering Algorithm

Traffic Processing Model of Big Data Base Station Based on Hybrid Improved CNN Algorithm and K-Centroids Clustering Algorithm

Crow Way: An Optimization Technique for generating the Weight and Bias in Deep CNN

A Heterogeneous Federated Transfer Learning Approach with Extreme Aggregation and Speed

Contact Info

Product

Resources

About