Privacy-enhanced momentum federated learning via differential privacy and chaotic system in industrial Cyber–Physical systems

Zhang, Zehui; Zhang, Linlin; Li, Qingdan; Wang, Kunshu; He, Ningxin; Gao, Tiegang

doi:10.1016/j.isatra.2021.09.007

Cited by 20 publications

(4 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Two of the most widely used privacy-preserving algorithms are homomorphic encryption [73,74,75,76] and multi-party computation [77,78,79]. On the other hand, differential privacy introduces random noise to either the data or the model parameters [80,81,82,83]. Although random noise is added to the data or model parameters, the algorithm provides statistical privacy guarantees while ensuring that the data or model parameters can still be used to facilitate effective global model development.…”

Section: ) Federated Learningmentioning

confidence: 99%

Enabling All In-Edge Deep Learning: A Literature Review

et al. 2023

View full text Add to dashboard Cite

In recent years, deep learning (DL) models have demonstrated remarkable achievements on non-trivial tasks such as speech recognition, image processing, and natural language understanding. One of the significant contributors to its success is the proliferation of end devices that acted as a catalyst to provide data for data-hungry DL models. However, computing DL training and inference is the main challenge. Usually, central cloud servers are used for the computation, but it opens up other significant challenges, such as high latency, increased communication costs, and privacy concerns. To mitigate these drawbacks, considerable efforts have been made to push the processing of DL models to edge servers (a mesh of computing devices near end devices). Moreover, the confluence point of DL and edge has given rise to edge intelligence (EI). International Electrotechnical Commission (IEC) defines EI as the concept where the data is acquired, stored, and processed utilizing edge computing with DL and advanced networking capabilities. Broadly, EI has six levels of categories based on where the training and inference of DL take place, e.g., cloud server, edge server and end devices. This survey paper focuses primarily on the fifth level of EI, called all in-edge level, where DL training and inference (deployment) are performed solely by edge servers. All in-edge is suitable when the end devices have low computing resources, e.g., Internet-of-Things, and other requirements such as latency and communication cost are important such as in mission-critical applications (e.g., health care). Besides, 5G/6G networks are envisioned to use all in-edge. Firstly, this paper presents all in-edge computing architectures, including centralized, decentralized, and distributed. Secondly, this paper presents enabling technologies, such as model parallelism, data parallelism, and split learning, which facilitates DL training and deployment at edge servers. Thirdly, model adaptation techniques based on model compression and conditional computation are described because the standard cloud-based DL deployment cannot be directly applied to all in-edge due to its limited computational resources. Fourthly, this paper discusses eleven key performance metrics to evaluate the performance of DL at all in-edge efficiently. Finally, several open research challenges in the area of all in-edge are presented. INDEX TERMS Artificial intelligence, all in-edge, deep learning, distributed systems, decentralized systems, edge intelligence I. INTRODUCTION T HE global community is increasingly becoming a datadriven environment in which end devices are generating vast quantities of data outside of the traditional data centers. International Telecommunication Union anticipates that global internet traffic per month will reach 607 Exabytes (EB) in 2025 and 5016 EB in 2030 [1]. This enormous amount of data has a positive impact on artificial intelligence (AI) applications. In particular, deep learning (DL) rely on the availability of large quantities of data for its d...

show abstract

Section: ) Federated Learningmentioning

confidence: 99%

Enabling All In-Edge Deep Learning: A Literature Review

et al. 2023

View full text Add to dashboard Cite

show abstract

“…Two of the most widely used privacy-preserving algorithms are homomorphic encryption [58,101,234,252] and multi-party computation [174,176,211,232]. On the other hand, differential privacy introduces random noise to the data as well as to the model parameters [69,256,309,310].…”

Section: Enabling Technologiesmentioning

confidence: 99%

Enabling Deep Learning for All-in EDGE paradigm

Joshi¹,

Hasanuzzaman²,

Thapa³

et al. 2022

Preprint

View full text Add to dashboard Cite

Deep Learning-based models have been widely investigated, and they have demonstrated significant performance on non-trivial tasks such as speech recognition, image processing, and natural language understanding. However, this is at the cost of substantial data requirements. Considering the widespread proliferation of edge devices (e.g., Internet of Things devices) over the last decade, Deep Learning in the edge paradigm, such as device-cloud integrated platforms, is required to leverage its superior performance. Moreover, it is suitable from the data requirements perspective in the edge paradigm because the proliferation of edge devices has resulted in an explosion in the volume of generated and collected data. However, there are difficulties due to other requirements such as high computation, high latency, and high bandwidth caused by Deep Learning applications in real-world scenarios. In this regard, this survey paper investigates Deep Learning at the edge, its architecture, enabling technologies, and model adaption techniques, where edge servers and edge devices participate in deep learning training and inference. For simplicity, we call this paradigm the All-in EDGE paradigm. Besides, this paper presents the key performance metrics for Deep Learning at the All-in EDGE paradigm to evaluate various deep learning techniques and choose a suitable design. Moreover, various open challenges arising from the deployment of Deep Learning at the All-in EDGE paradigm are identified and discussed.

show abstract

“…As a distributed training method, federated learning (FL) can cooperate with multiple clients to develop a global model while ensuring user privacy and avoiding data sharing [9]. It provides a feasible solution for solving the above problems and has achieved excellent results in many fields, such as mechanical fault diagnosis [10,11].…”

Section: Introductionmentioning

confidence: 99%

A novel federated deep learning framework for diagnosis of partial discharge in gas-insulated switchgear

Wang¹,

Yan²,

Jing³

et al. 2022

Meas. Sci. Technol.

View full text Add to dashboard Cite

In recent years, many different deep learning methods have been developed to ensure the safe and stable operation of gas-insulated switchgear (GIS). However, the use of these methods to achieve excellent results depends on obtaining as much training data as possible, which is difficult to accomplish because of conflicts of interest among different clients and privacy concerns. To address this issue, this paper proposes a novel federated deep learning (FDL) for the diagnosis of partial discharge (PD) in GIS. A federated learning (FL) based on an improved federated averaging algorithm is proposed, which allows different clients to collaboratively participate in model training and preserves data privacy. In addition, a novel subtractive attention Siamese network is introduced for feature extraction and classification, which achieves the high-precision classification of unbalanced data. Experimental results showed that the diagnostic accuracy of the proposed FDL reached 95.61%, which was significantly higher than that achieved by other methods. The proposed FDL can also achieve excellent performance in the case of unbalanced samples and small samples. As a distributed learning, FL does not require clients to share data, and clients can collaboratively develop an effective global diagnostic model, which provides a novel solution for GIS PD diagnosis.

show abstract

Privacy-enhanced momentum federated learning via differential privacy and chaotic system in industrial Cyber–Physical systems

Cited by 20 publications

References 36 publications

Enabling All In-Edge Deep Learning: A Literature Review

Enabling All In-Edge Deep Learning: A Literature Review

Enabling Deep Learning for All-in EDGE paradigm

A novel federated deep learning framework for diagnosis of partial discharge in gas-insulated switchgear

Contact Info

Product

Resources

About