Dynamic RAN Slicing for Service-Oriented Vehicular Networks via Constrained Learning

Wu, Wen; Chen, Nan; Zhou, Conghao; Li, Mushu; Shen, Xuemin; Zhuang, Weihua; Li, Xu

doi:10.1109/jsac.2020.3041405

Cited by 135 publications

(36 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In a small-timescale operation stage, certain types of tasks generated from active AVs in each scheduling slot are offloaded to selected BSs, and radio resources on the BSs are also allocated to each active AV for task transmission. As the length of each planning window is usually in a large timescale (e.g., hours), the average number of AVs in road zone z of S k (k = 0, 1, ..., n), denoted by M k,z , within one window time is assumed constant [20], but can vary between consecutive windows to reflect large-timescale vehicular traffic load dynamics. A macroscopic fluid-flow vehicular mobility model [23] is adopted to describe the relation between the mean velocity for AVs in zone z of S k , denoted by v k,z , and M k,z in one planning window, given by…”

Section: System Model a Network Modelmentioning

confidence: 99%

“…There exist studies on how to slice the radio resources for supporting different vehicular services, by maximizing 1) the number of admitted service requests [17], 2) the combination of spectrum efficiency and service satisfaction ratio [18], or 3) the satisfaction of service delay and information freshness [19]. Joint RAN slicing and computing workload allocation is studied in [20], for minimizing the overall system cost of allocating the two-dimensional resources with task offloading delay and queueing stability constraints.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Joint RAN Slicing and Computation Offloading for Autonomous Vehicular Networks: A Learning-Assisted Hierarchical Approach

Shi

et al. 2021

IEEE Open J. Veh. Technol.

Self Cite

View full text Add to dashboard Cite

In this paper, a two-timescale radio access network (RAN) slicing and computing task offloading problem is investigated for a cloud-enabled autonomous vehicular network (C-AVN). We aim at jointly maximizing the communication and computing resource utilization with diverse quality-of-service (QoS) guarantee for autonomous driving tasks. Specifically, to capture the small-timescale network dynamics, a computing task scheduling problem is formulated as a stochastic optimization program, for maximizing the long-term network-wide computation load balancing with minimum task offloading variations. Due to the large problem size and unavailable network state transition probabilities, we employ cooperative multi-agent deep Q-learning (MA-DQL) with fingerprint to solve the problem by learning the set of stationary task offloading policies with stabilized convergence. Given the task offloading decisions, we further study a RAN slicing problem in a large timescale, which is formulated as a convex optimization program. We focus on optimizing the radio resource slicing ratios among base stations, to maximize the aggregate network utility with statistical QoS provisioning for autonomous driving tasks. Based on the impact of radio resource slicing on computation load balancing, we propose a two-timescale hierarchical optimization framework to maximize both communication and computing resource utilization. Extensive simulation results are provided to demonstrate the effectiveness of the proposed framework in comparison with state-of-the-art schemes.Index Terms-Autonomous vehicular networks, computing task offloading, RAN slicing, resource sharing, task scheduling, cooperative multi-agent deep Q-learning, learning-assisted hierarchical approach, diverse QoS provisioning.

show abstract

Section: System Model a Network Modelmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Joint RAN Slicing and Computation Offloading for Autonomous Vehicular Networks: A Learning-Assisted Hierarchical Approach

Shi

et al. 2021

IEEE Open J. Veh. Technol.

Self Cite

View full text Add to dashboard Cite

show abstract

“…where λ k+1 = (β 0 + 1) g k+1 g k . Next, we analyze the characteristics of the transmission power by giving (7) and (8).…”

Section: B C-p Efficiencymentioning

confidence: 99%

“…Remark 2 (8) indicates that the access capacity depends on P max , β 0 and the channel gain g 0 . A higher transmission power of vehicle, a higher system achievable capacity is achieved, while the SINR of vehicles and the position of UAV are fixed.…”

Section: The Transmission Power Of Plmentioning

confidence: 99%

Control Efficient Power Allocation of Uplink NOMA in UAV-Aided Vehicular Platooning

Sun¹,

Tang

2021

IEEE Access

View full text Add to dashboard Cite

Power allocation in non-orthogonal multiple access (NOMA) systems is essential to avoid multi-user detection failure. However, controlling the uplink transmission power brings extra signaling overhead, especially in dynamic environments. In this paper, we investigate the uplink transmission power allocation problem in an unmanned aerial vehicle (UAV)-aided platooning NOMA system, where the UAV acts as a relay to assist inter-vehicle communications. Platoon vehicles are admitted to the same vehicle-to-UAV (V2U) channel via our proposed sequence-based power allocation (SPA) strategy. Instead of achieving the optimal energy efficiency, our goal is to design a simple yet effective power control scheme. Specially, in SPA strategy, vehicles determine their transmission power according to their positions in the platoon. Considering platoon dynamics due to vehicle mobility, a power-reserved mechanism is further incorporated in the proposed strategy to pre-allocate power resources for newly joined vehicles. In addition, a power-moving mechanism is further employed to guarantee the uplink signal-to-interferenceplus-noise ratio (SINR) of vehicles by changing vehicle transmission power into higher levels during vehicle leaving a platoon. As the system only needs to update vehicle positions for the SPA strategy, only a small amount of signaling is required in our power allocation strategy. Extensive simulation results are provided to demonstrate that the proposed SPA strategy ensures the required SINR of platoon communication in different dynamic scenarios.INDEX TERMS UAV relay, vehicle platooning, non-orthogonal multiple access, power allocation.

show abstract

“…However, they cannot address constrained action space in edge computing. Wu et al [14] use sof tmax to capture the task partitioning actions to satisfy the constraints of the action space, which is a proportional action space, the sum of which should be one. However, sof tmax does not have an exploration mechanism to explore all the possible actions and derive the optimal policies.…”

Section: Introductionmentioning

confidence: 99%

D3PG: Dirichlet DDGP for Task Partitioning and Offloading with Constrained Hybrid Action Space in Mobile Edge Computing

Ale¹,

King²,

Zhang³

et al. 2021

Preprint

View full text Add to dashboard Cite

<div> Mobile Edge Computing (MEC) has been regarded as a promising paradigm to reduce service latency for data processing in Internet of Things, by provisioning computing resources at network edge. In this work, we jointly optimize the task partitioning and computational power allocation for computation offloading in a dynamic environment with multiple IoT devices and multiple edge servers. We formulate the problem as a Markov decision process with constrained hybrid action space, which cannot be well handled by existing deep reinforcement learning (DRL) algorithms. Therefore, we develop a novel Deep Reinforcement Learning called Dirichlet Deep Deterministic Policy Gradient (D3PG), which </div><div>is built on Deep Deterministic Policy Gradient (DDPG) to solve the problem. The developed model can learn to solve multi-objective optimization, including maximizing the number of tasks processed before expiration and minimizing the energy cost and service latency. More importantly, D3PG can effectively deal with constrained distribution-continuous hybrid action space, where the distribution variables are for the task partitioning and offloading, while the continuous variables are for computational frequency control. Moreover, the D3PG can address many similar issues in MEC and general reinforcement learning problems. Extensive simulation results show that the proposed D3PG outperforms the state-of-art methods.</div><div> Mobile Edge Computing (MEC) has been regarded as a promising paradigm to reduce service latency for data processing in Internet of Things, by provisioning computing resources at network edge. In this work, we jointly optimize the task partitioning and computational power allocation for computation offloading in a dynamic environment with multiple IoT devices and multiple edge servers. We formulate the problem as a Markov decision process with constrained hybrid action space, which cannot be well handled by existing deep reinforcement learning (DRL) algorithms. Therefore, we develop a novel Deep Reinforcement Learning called Dirichlet Deep Deterministic Policy Gradient (D3PG), which is built on Deep Deterministic Policy Gradient (DDPG) to solve the problem. The developed model can learn to solve multi-objective optimization, including maximizing the number of tasks processed before expiration and minimizing the energy cost and service latency. More importantly, D3PG can effectively deal with constrained distribution-continuous hybrid action space, where the distribution variables are for the task partitioning and offloading, while the continuous variables are for computational frequency control. Moreover, the D3PG can address many similar issues in MEC and general reinforcement learning problems. Extensive simulation results show that the proposed D3PG outperforms the state-of-art methods.</div>

show abstract

Dynamic RAN Slicing for Service-Oriented Vehicular Networks via Constrained Learning

Cited by 135 publications

References 38 publications

Joint RAN Slicing and Computation Offloading for Autonomous Vehicular Networks: A Learning-Assisted Hierarchical Approach

Joint RAN Slicing and Computation Offloading for Autonomous Vehicular Networks: A Learning-Assisted Hierarchical Approach

Control Efficient Power Allocation of Uplink NOMA in UAV-Aided Vehicular Platooning

D3PG: Dirichlet DDGP for Task Partitioning and Offloading with Constrained Hybrid Action Space in Mobile Edge Computing

Contact Info

Product

Resources

About