Dynamic Sampling and Selective Masking for Communication-Efficient Federated Learning

Ji, Shaoxiong; Jiang, Wenqi; Walid, Anwar; Li, Xue

doi:10.1109/mis.2021.3114610

Cited by 38 publications

(11 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The approach of Ji et al [22] is to progressively decrease the fraction of clients that perform local computation, while reducing the amount of transmitted data by means of a mechanism that masks part of the parameters of local models. Sparse Ternary Compression (STC) is a protocol proposed by Sattler et al [23] to compress upstream and downstream communications between server and clients.…”

Section: Efficient Flmentioning

confidence: 99%

FLAD: Adaptive Federated Learning for DDoS Attack Detection

Doriguzzi-Corin¹,

Siracusa²

2022

Preprint

View full text Add to dashboard Cite

Federated Learning (FL) has been recently receiving increasing consideration from the cybersecurity community as a way to collaboratively train deep learning models with distributed profiles of cyberthreats, with no disclosure of training data. Nevertheless, the adoption of FL in cybersecurity is still in its infancy, and a range of practical aspects have not been properly addressed yet. Indeed, the Federated Averaging algorithm at the core of the FL concept requires the availability of test data to control the FL process. Although this might be feasible in some domains, test network traffic of newly discovered attacks cannot be always shared without disclosing sensitive information.In this paper, we address the convergence of the FL process in dynamic cybersecurity scenarios, where the trained model must be frequently updated with new recent attack profiles to empower all members of the federation with latest detection features. To this aim, we propose FLAD (adaptive Federated Learning Approach to DDoS attack detection), a FL solution for cybersecurity applications based on an adaptive mechanism that orchestrates the FL process by dynamically assigning more computation to those members whose attacks profiles are harder to learn, without the need of sharing any test data to monitor the performance of the trained model. Using a recent dataset of DDoS attacks, we demonstrate that FLAD outperforms the original FL algorithm in terms of convergence time and accuracy across a range of unbalanced datasets of heterogeneous DDoS attacks. We also show the robustness of our approach in a realistic scenario, where we retrain the deep learning model multiple times to introduce the profiles of new attacks on a pre-trained model.

show abstract

Section: Efficient Flmentioning

confidence: 99%

FLAD: Adaptive Federated Learning for DDoS Attack Detection

Doriguzzi-Corin¹,

Siracusa²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Nevertheless, vanilla -synch-based methods add extra load on the underlying communication system, as they will ask all the devices to upload their data, and the master node starts its update with the first received data. Reference [14] proposed an algorithm to adjust at every iteration. References [6], [13], [15] proposed various approaches to eliminate some unnecessary uploads.…”

Section: A Distributed Learningmentioning

confidence: 99%

Interplay between Distributed AI Workflow and URLLC

Ganjalizadeh¹,

Ghadikolaei²,

Haraldson³

et al. 2022

Preprint

View full text Add to dashboard Cite

Distributed artificial intelligence (AI) has recently accomplished tremendous breakthroughs in various communication services, ranging from fault-tolerant factory automation to smart cities. When distributed learning is run over a set of wireless connected devices, random channel fluctuations, and the incumbent services simultaneously running on the same network affect the performance of distributed learning. In this paper, we investigate the interplay between distributed AI workflow and ultra-reliable low latency communication (URLLC) services running concurrently over a network. Using 3GPP compliant simulations in a factory automation use case, we show the impact of various distributed AI settings (e.g., model size and the number of participating devices) on the convergence time of distributed AI and the application layer performance of URLLC. Unless we leverage the existing 5G-NR quality of service handling mechanisms to separate the traffic from the two services, our simulation results show that the impact of distributed AI on the availability of the URLLC devices is significant. Moreover, with proper setting of distributed AI (e.g., proper user selection), we can substantially reduce network resource utilization, leading to lower latency for distributed AI and higher availability for the URLLC users. Our results provide important insights for future 6G and AI standardization.

show abstract

“…With the development of artificial intelligence, the structure of deep learning network becomes more and more complex. As the complexity of the network model increases, the number of parameters will also increase [23]. Federated learning mainly transmits model parameters, and the large number of parameters in existing deep learning models may cause the failure of federated learning.…”

Section: Introductionmentioning

confidence: 99%

Improved lightweight federated learning network for fault feature extraction of reciprocating machinery

Zhang,

Duan,

et al. 2024

Meas. Sci. Technol.

View full text Add to dashboard Cite

he working environment of reciprocating machinery is complex, characterized by nonlinear and non-stationary signals. Deep learning can solve the above problems, but it has its own problems such as complex model and large amount of parameters. Additionally, privacy considerations among enterprises prevent data sharing, leading to the emergence of "data islands" and inadequate training of deep learning models. Based on the above analysis, this paper proposes a reciprocating mechanical feature extraction method based on an improved federated lightweight network. A lightweight network SqueezeNet model is used to solve the problems such as long training time of deep learning. By establishing a federated learning framework, the reciprocating mechanical data can be collectively diagnosed across various enterprises, thereby addressing the problem of limited model training caused by insufficient data. Furthermore, to enhance the accuracy of network training and diagnosis, modifications are made to the SqueezeNet network to reduce the number of model parameters while increasing the number and variety of feature extractions. Experimental results demonstrate that when the number of 1×1 and 3×3 channels is 1 to 7, the fault diagnosis accuracy is the highest, about 97.96%, which enriches the categories of feature extraction. The number of parameters in In-SqueezeNet is 56% of that in SqueezeNet network model, and the training time is reduced by nearly 15%. The fault diagnosis accuracy is increased from 95.1% to 97.3%, and the diversity of extracted features is increased. Compared with other network models such as ResNet, the improved lightweight federated learning network has a fault diagnosis accuracy of 96.6%, an improvement of 10.6%. At the same time, the training time was reduced to 1982s, a reduction of about 41.5%. The validity of the proposed model is further verified.

show abstract

Dynamic Sampling and Selective Masking for Communication-Efficient Federated Learning

Cited by 38 publications

References 7 publications

FLAD: Adaptive Federated Learning for DDoS Attack Detection

FLAD: Adaptive Federated Learning for DDoS Attack Detection

Interplay between Distributed AI Workflow and URLLC

Improved lightweight federated learning network for fault feature extraction of reciprocating machinery

Contact Info

Product

Resources

About