Secure Distributed On-Device Learning Networks with Byzantine Adversaries

Abstract

The privacy concern exists when the central server has the copies of datasets. Hence, there is a paradigm shift for the learning networks to change from centralized in-cloud learning to distributed on-device learning. Benefit from the parallel computing, the on-device learning networks have a lower bandwidth requirement than the in-cloud learning networks. Moreover, the on-device learning networks also have several desirable characteristics such as privacy preserving and flexibility. However, the on-device learning networks are vulnerable to the malfunctioning terminals across the networks. The worst-case malfunctioning terminals are the Byzantine adversaries, that can perform arbitrary harmful operations to compromise the learned model based on the full knowledge of the networks. Hence, the design of secure learning algorithms becomes an emerging topic in the on-device learning networks with Byzantine adversaries. In this article, we present a comprehensive overview of the prevalent secure learning algorithms for the two promising on-device learning networks: Federated-Learning networks and decentralized-learning networks. We also review several future research directions in the Federated-Learning and decentralized-learning networks.

Green IoT and Edge AI as Key Technological Enablers for a Sustainable Digital Transition towards a Smart Circular Economy: An Industry 5.0 Use Case

“…• Additional mechanisms are needed to offer protection against network, physical, software, and encryption attacks. In addition, it is critical to have protection against adversarial attacks during on-device learning [130]. • Future communications networks.…”

Collective Learning: A 10-Year Odyssey to Human-centered Distributed Intelligence

“…Or agents selecting plans that maximize the inefficiency cost or even agents that arbitrary violate the execution of their selected plans. Making collective learning tolerant to Byzantine faults may require methods to identify and isolate such agents or novel collective actions by other agents to remedy the effect of adversary behavior [61].…”

Byzantine-Resilient Federated Machine Learning via Over-the-Air Computation

“…FL avoids the direct raw data exchange among edge devices to alleviate the privacy concerns while collaboratively training a common model under the orchestration of a central server. However, a number of challenges arise for the practical deployment of FL, including the statistical challenges with non-IID (not independent and identically distributed) datasets across edge devices [6], [8], [9], high communication costs during the training process [6], [8], privacy and security concerns because of adversarial devices [3], [9], [10], heterogeneous devices with varying resource constraints [9], and system design issues [11], such as the unreliable device connectivity, interrupted execution and slow convergence.…”

“…The state-of-the-art research progress has been made on secure FL in presence of Byzantine devices. The authors in [10] classified the current secure FL algorithms into four categories: robust aggregation rule [10], [21], preprocess method from the information-the-oretical perspective [22], [23], model with regularization term [24], and adversarial detection [25], [26]. In this paper, we focus on the robust aggregation rule.…”