RJCC: Reinforcement-Learning-Based Joint Communicational-and-Computational Resource Allocation Mechanism for Smart City IoT

Xu, Siya; Liu, Qingchuan; Gong, Bei; Qi, Feng; Guo, Shaoyong; Qiu, Xuesong; Yang, Chao

doi:10.1109/jiot.2020.3002427

“…Specifically, based on primal/dual/hierarchical/partial decomposition [83], the original optimization problem is first decoupled into several subproblems (usually two subproblems, such as the offloading decision subproblem and resource allocation subproblem for computational offloading optimization [84], [85]). Then, some with large-scale uncertain states and action spaces are solved by RL algorithms with high efficiency, and other subproblems with determined network states are solved by conventional algorithms, such as conventional optimization algorithms [63], [85]- [89], searching algorithms [84], [90], [91], heuristic algorithms [92], and game theory [93], [94]. Alternatively, integrating RL and machine learning algorithms is adopted, such as supervised/unsupervised learning [95]- [97], federated learning (FL) [98]- [100], and hierarchical RL [67], [101]- [104], for optimization efficiency improvement.…”

Section: B Reinforcement Learning-empowered Mecmentioning

confidence: 99%

Reinforcement Learning-Empowered Mobile Edge Computing for 6G Edge Intelligence

Wei

¹

,

Guo

²

,

Li

³

et al. 2022

View full text Add to dashboard Cite

Mobile edge computing (MEC) is considered a novel paradigm for computation-intensive and delay-sensitive tasks in fifth generation (5G) networks and beyond. However, its uncertainty, referred to as dynamic and randomness, from the mobile device, wireless channel, and edge network sides, results in high-dimensional, nonconvex, nonlinear, and NP-hard optimization problems. Thanks to the evolved reinforcement learning (RL), upon iteratively interacting with the dynamic and random environment, its trained agent can intelligently obtain the optimal policy in MEC. Furthermore, its evolved versions, such as deep reinforcement learning (DRL), can achieve higher convergence speed efficiency and learning accuracy based on the parametric approximation for the large-scale state-action space. This paper provides a comprehensive research review on RL-enabled MEC and offers insight for development in this area. More importantly, associated with free mobility, dynamic channels, and distributed services, the MEC challenges that can be solved by different kinds of RL algorithms are identified, followed by how they can be solved by RL solutions in diverse mobile applications. Finally, the open challenges are discussed to provide helpful guidance for future research in RL training and learning MEC. INDEX TERMS Mobile edge computing (MEC), network uncertainty, reinforcement learning (RL).

show abstract

“…searching algorithms [79], [85], [86], heuristic algorithms [87], and game theory [88], [89]. Alternatively, integrating RL and machine learning algorithms is adopted, such as supervised/unsupervised learning [90]- [92], federated learning (FL) [93]- [95], and hierarchical RL [64], [96]- [99], for optimization efficiency improvement.…”

Section: B Reinforcement Learning-empowered Mecmentioning

confidence: 99%

“…In addition to vehicular and industrial networks, IoT networks also include many specific scenarios, such as smart cities [65], [86], [167], [168], satellite-aided networks [81], [82], and narrowband (NB)-IoT [169], [170]. For computationintensive device access and delay-sensitive service requirements, MEC-based SDN, NFV, routing, and narrowband cellular transmission technologies provide centralized control, heterogeneous resource management, and effective communication.…”

Section: General Iot Applicationsmentioning

confidence: 99%

“…For computationintensive device access and delay-sensitive service requirements, MEC-based SDN, NFV, routing, and narrowband cellular transmission technologies provide centralized control, heterogeneous resource management, and effective communication. In [86], cloud-edge-terminal collaboration was considered in MEC-based smart city IoT networks. In particular, a joint communication and computational resource allocation problem is formulated based on the Lyapunov drift plus penalty-based optimization theory to minimize the cost of total system delay subject to limited energy.…”

Section: General Iot Applicationsmentioning

confidence: 99%

See 1 more Smart Citation

Reinforcement Learning-Empowered Mobile Edge Computing for 6G Edge Intelligence

Wei,

Guo,

Li

et al. 2022

Preprint

0

View full text Add to dashboard Cite

Mobile edge computing (MEC) is considered a novel paradigm for computation-intensive and delay-sensitive tasks in fifth generation (5G) networks and beyond. However, its uncertainty, referred to as dynamic and randomness, from the mobile device, wireless channel, and edge network sides, results in high-dimensional, nonconvex, nonlinear, and NP-hard optimization problems. Thanks to the evolved reinforcement learning (RL), upon iteratively interacting with the dynamic and random environment, its trained agent can intelligently obtain the optimal policy in MEC. Furthermore, its evolved versions, such as deep RL (DRL), can achieve higher convergence speed efficiency and learning accuracy based on the parametric approximation for the large-scale state-action space. This paper provides a comprehensive research review on RL-enabled MEC and offers insight for development in this area. More importantly, associated with free mobility, dynamic channels, and distributed services, the MEC challenges that can be solved by different kinds of RL algorithms are identified, followed by how they can be solved by RL solutions in diverse mobile applications. Finally, the open challenges are discussed to provide helpful guidance for future research in RL training and learning MEC. Index Terms-Mobile edge computing (MEC), network uncertainty, reinforcement learning (RL). emote Areas irplane taverse vers Diverse Scenarios MEC Deployment Future Applications Smart City Smart Home Hazard/Remote Areas Smart Agriculture Digital Twin Holographic Communication AR/VR/XR Metaverse Metaverse Metavers Autonomous Driving Small Data Center Switch BS Traffic Light Satellite Ship Airplane S ll S it h BS T ffi Li ht S t llit Shi Ai l 5G Networks and Beyond 5G Networks and Beyond Smart Factory performance. Its typical algorithms are K-means clustering, principal component analysis, and independent component analysis, which can be used in small cell clustering, heterogeneous network clustering, smart grid user classification, etc. However, in unreliable radio network environments, the classification accuracy decreases, which readily causes slow and inaccurate actions in MEC systems. • Different from the static solutions provided by supervised learning and unsupervised learning, RL gives a constantly evolving intelligent framework [21]-[27]. In RL [29], an agent is enabled to make proper decisions based on frequent interactions with the stochastic and dynamic environment. Based on the Markov models, the RL operates in a feedback mechanism (a closed loop) without the knowledge of input data, where based on the previous and current states, the agent can execute its actions to maximize the reward function. Additionally, the above behaviors, including the states, actions, and rewards, accumulate to generate experiences. The classic RL algorithm is Q-learning, which suffers from the curse of dimensionality caused by the large dimensions of the state-action spaces. Accordingly, upon leveraging the low-dimensional representation for the high-dimensional state-ac...

show abstract

“…This means that, a constant computing intensity (in computing cycles per unit data) exists for such tasks, and we can use it to capture the effective computing capability of a specific device. Existing literature, such as [30]- [32], has leveraged this observation to characterize deep learning workloads, and in this work, we adopt it to estimate the computing cycles amount given the partitions and DNN layers. Concretely, in Eq.…”

Section: A Problem Formulationmentioning

confidence: 99%

CoEdge: Cooperative DNN Inference with Adaptive Workload Partitioning over Heterogeneous Edge Devices

Zeng¹,

Chen²,

Zhou³

et al. 2020

Preprint

0

View full text Add to dashboard Cite

Recent advances in artificial intelligence have driven increasing intelligent applications at the network edge, such as smart home, smart factory, and smart city. To deploy computationally intensive Deep Neural Networks (DNNs) on resourceconstrained edge devices, traditional approaches have relied on either offloading workload to the remote cloud or optimizing computation at the end device locally. However, the cloud-assisted approaches suffer from the unreliable and delay-significant widearea network, and the local computing approaches are limited by the constrained computing capability. Towards high-performance edge intelligence, the cooperative execution mechanism offers a new paradigm, which has attracted growing research interest recently. In this paper, we propose CoEdge, a distributed DNN computing system that orchestrates cooperative DNN inference over heterogeneous edge devices. CoEdge utilizes available computation and communication resources at the edge and dynamically partitions the DNN inference workload adaptive to devices' computing capabilities and network conditions. Experimental evaluations based on a realistic prototype show that CoEdge outperforms status-quo approaches in saving energy with close inference latency, achieving up to 25.5%∼66.9% energy reduction for four widely-adopted CNN models.

show abstract

RJCC: Reinforcement-Learning-Based Joint Communicational-and-Computational Resource Allocation Mechanism for Smart City IoT

Cited by 33 publications

References 48 publications

Reinforcement Learning-Empowered Mobile Edge Computing for 6G Edge Intelligence

Reinforcement Learning-Empowered Mobile Edge Computing for 6G Edge Intelligence

Reinforcement Learning-Empowered Mobile Edge Computing for 6G Edge Intelligence

CoEdge: Cooperative DNN Inference with Adaptive Workload Partitioning over Heterogeneous Edge Devices

Contact Info

Product

Resources

About