Joint Demand Forecasting and DQN-Based Control for Energy-Aware Mobile Traffic Offloading

Huang, Chih‐Wei; Chen, Po Chen

doi:10.1109/access.2020.2985679

Cited by 19 publications

(11 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Also, some important parameters are listed in Table 2. Moreover, In (28 further improve the learning efficiency. Furthermore, when dividing the computational resources of each FAP, in order to avoid missing the optimal solution, we set that each resource block is 0.1 Figures 5 and 6 display the learning curves representing the accumulated reward obtained by the agent in each training epoch of the DDQN algorithm and the DQN algorithm, respectively.…”

Section: Resultsmentioning

confidence: 99%

“…Environment & State: The environment in RL is defined as the set of all possible states, and the essence of RL is to perform actions to cause the state transfer [28]. Therefore, we set a matrix S as the state, which has the same shape as the matrix P , and the value of s ij in the matrix S should only be 0 or 1, s ij = 1 represents the agent selects FAP i (or DCN i) for UE j, otherwise s ij = 0 .…”

Section: Markov Decision Processmentioning

confidence: 99%

See 1 more Smart Citation

A reinforcement learning-based computing offloading and resource allocation scheme in F-RAN

Jiang

Gao

et al. 2021

EURASIP J. Adv. Signal Process.

View full text Add to dashboard Cite

This paper investigates a computing offloading policy and the allocation of computational resource for multiple user equipments (UEs) in device-to-device (D2D)-aided fog radio access networks (F-RANs). Concerning the dynamically changing wireless environment where the channel state information (CSI) is difficult to predict and know exactly, we formulate the problem of task offloading and resource optimization as a mixed-integer nonlinear programming problem to maximize the total utility of all UEs. Concerning the non-convex property of the formulated problem, we decouple the original problem into two phases to solve. Firstly, a centralized deep reinforcement learning (DRL) algorithm called dueling deep Q-network (DDQN) is utilized to obtain the most suitable offloading mode for each UE. Particularly, to reduce the complexity of the proposed offloading scheme-based DDQN algorithm, a pre-processing procedure is adopted. Then, a distributed deep Q-network (DQN) algorithm based on the training result of the DDQN algorithm is further proposed to allocate the appropriate computational resource for each UE. Combining these two phases, the optimal offloading policy and resource allocation for each UE are finally achieved. Simulation results demonstrate the performance gains of the proposed scheme compared with other existing baseline schemes.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Markov Decision Processmentioning

confidence: 99%

A reinforcement learning-based computing offloading and resource allocation scheme in F-RAN

Jiang

Gao

et al. 2021

EURASIP J. Adv. Signal Process.

View full text Add to dashboard Cite

show abstract

“…Deep Q network (DQN) was introduced in Huang et al [19] for forecasting traffic under high traffic loads. The designed network was not capable of reducing time and memory consumptions.…”

Section: Related Researchmentioning

confidence: 99%

Network Traffic Prediction Using Radial Kernelized-Tversky Indexes-Based Multilayer Classifier

Govindarajan¹,

Chandrasekaran²,

Pillai³

2022

Computer Systems Science and Engineering

View full text Add to dashboard Cite

Accurate cellular network traffic prediction is a crucial task to access Internet services for various devices at any time. With the use of mobile devices, communication services generate numerous data for every moment. Given the increasing dense population of data, traffic learning and prediction are the main components to substantially enhance the effectiveness of demand-aware resource allocation. A novel deep learning technique called radial kernelized LSTM-based connectionist Tversky multilayer deep structure learning (RKLSTM-CTMDSL) model is introduced for traffic prediction with superior accuracy and minimal time consumption. The RKLSTM-CTMDSL model performs attribute selection and classification processes for cellular traffic prediction. In this model, the connectionist Tversky multilayer deep structure learning includes multiple layers for traffic prediction. A large volume of spatial-temporal data are considered as an input-to-input layer. Thereafter, input data are transmitted to hidden layer 1, where a radial kernelized long short-term memory architecture is designed for the relevant attribute selection using activation function results. After obtaining the relevant attributes, the selected attributes are given to the next layer. Tversky index function is used in this layer to compute similarities among the training and testing traffic patterns. Tversky similarity index outcomes are given to the output layer. Similarity value is used as basis to classify data as heavy network or normal traffic. Thus, cellular network traffic prediction is presented with minimal error rate using the RKLSTM-CTMDSL model. Comparative evaluation proved that the RKLSTM-CTMDSL model outperforms conventional methods.

show abstract

“…Environment & State: The environment in RL is defined as the set of all possible states, and the essence of RL is to perform actions to cause the state transfer [24]. Therefore, we set a matrix S as the state, which has the same shape as the matrix P , and the value of s ij in the matrix S should only be 0 or 1, s ij = 1 represents the agent selects FAP i (or DCN i) for UE j, otherwise s ij = 0.…”

Section: ) Markov Decision Processmentioning

confidence: 99%

“…Where ω and ϕ are the network parameters for V (s t ) and A(s t , a t ), respectively. Specifically, V (s t ) stands for the excepted accumulated reward at the state s t , and A(s t , a t ) indicates the degree of superiority of action a t over the average level in state s t presented as formula (24) and (25).…”

Section: ) Ddqn Algorithmmentioning

confidence: 99%

A Reinforcement Learning based Computing Offloading and Resource Allocation Scheme in F-RAN

Jiang

Gao

2021

Preprint

View full text Add to dashboard Cite

This paper investigates a computing offloading policy and the allocation of computational resource for multiple user equipments (UEs) in Device-to-Device (D2D) aided fog radio access networks (F-RANs). Concerning the dynamically changing wireless environment where the channel state information (CSI) is difficult to predict and know exactly, we formulate the problem of task offloading and resource optimization as a mixed-integer nonlinear programming problem to maximize the total utility of all UEs. Concerning the non-convex property of the formulated problem, we decouple the original problem into two phases to solve. Firstly, a centralized deep reinforcement learning (DRL) algorithm called Dueling Deep Q-Network (DDQN) is utilized to obtain the most suitable offloading mode for each UE. Particularly, to reduce the complexity of the proposed offloading scheme based DDQN algorithm, a pre-processing procedure is adopted. Then a distributed Deep Q-Network (DQN)algorithm based on the training result of the DDQN algorithm is further proposed to allocate the appropriate computational resource for each UE. Combining these two phases, the optimal offloading policy and resource allocation for each UE are finally achieved. Simulation results demonstrate the performance gains of the proposed scheme compared with other existing baseline schemes.

show abstract

Joint Demand Forecasting and DQN-Based Control for Energy-Aware Mobile Traffic Offloading

Cited by 19 publications

References 35 publications

A reinforcement learning-based computing offloading and resource allocation scheme in F-RAN

A reinforcement learning-based computing offloading and resource allocation scheme in F-RAN

Network Traffic Prediction Using Radial Kernelized-Tversky Indexes-Based Multilayer Classifier

A Reinforcement Learning based Computing Offloading and Resource Allocation Scheme in F-RAN

Contact Info

Product

Resources

About