Double Deep Q-Network for Power Allocation in Cloud Radio Access Network

Iqbal, Amjad; Tham, Mau‐Luen; Chang, Yoong Choon

doi:10.1109/ccet50901.2020.9213138

Cited by 15 publications

(12 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…3) In the RL approach, each action is matched to a corresponding reward, and the system learns the optimal actions that lead to the greatest accumulation of rewards. [12] devised a double deep Q-network-based resource allocation method that minimizes the total power consumption subject to the constraints on the transmit power of each remote radio head (RRH) and user rates in the cloud radio access network. [13] is an extension of [12] which focuses on energy efficiency maximization instead of power minimization.…”

Section: A Motivationmentioning

confidence: 99%

Proactive Power Control and Position Deployment for Drone Small Cells: Joint Supervised and Unsupervised Learning

Cheng¹,

Shih

Chang

2021

IEEE Access

View full text Add to dashboard Cite

Since unmanned aerial vehicles (UAV) are easily deployed, highly mobile, and hover capability, they are utilized for many commercial applications. In particular, small cells mounted on UAVs, also known as drone small cells (DSC), may provide temporary relief or ancillary programs for the wireless network. In this paper, we design a prediction-based proactive drone management (P 2 DM) framework to reduce network interference and improve energy efficiency in the multiple DSCs scenario. The P 2 DM framework can be divided into offline and online phases. In the offline phase, supervised learning is used to build a highly accurate mobility prediction model according to the historical data. The prediction model is launched in the online phase to predict the user position only using a small sample set. The system proactively determines whether a DSC should be awake or asleep at the next timeslot due to the predicted user positions. Since DSC has more longer awake time in the deep sleeping mode, it is previously awoken to avoid data propagation delay. To further overcome the difficulty of obtaining the key performance indicator data (i.e., labeled data) in the online phase, an unsupervised learning technique is employed for DSC repositioning and power control to improve energy efficiency. Our simulation results show that the P 2 DM framework can demonstrate the advantage in terms of execution time and energy efficiency compared to the existing method based on genetic algorithm (i.e., a heuristic algorithm).INDEX TERMS Data driven, supervised learning, unsupervised learning, mobility prediction, and Drone small cells.

show abstract

Section: A Motivationmentioning

confidence: 99%

Proactive Power Control and Position Deployment for Drone Small Cells: Joint Supervised and Unsupervised Learning

Cheng¹,

Shih

Chang

2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Iqbal et al in [100] investigated the energy efficiency maximization problem in C-RAN maintaining the per-RRH transmission power and user data rate constraints. Unlike the previously discussed studies, this work utilized a double DQN (DDQN) which incorporates an additional target DQN with the main DQN.…”

Section: A Power Consumption Optimizationmentioning

confidence: 99%

“…The proposed DDQN-based DRL model in [100] was implemented with an FFNN containing two hidden layers with 64 and 32 neuron. ReLU was used as the activation function and an experience replay with capacity 500 was considered.…”

Section: Evaluation Techniques For Dl-based C-ranmentioning

confidence: 99%

A Survey on Applications of Deep Learning in Cloud Radio Access Network

Rodoshi

Choi

2021

IEEE Access

View full text Add to dashboard Cite

The necessity for high-speed and low-latency connectivity of the vast number of mobile users is rising with the immense usage of mobile applications. A cloud radio access network (C-RAN) is a promising framework for next-generation cellular communication, which can satisfy the requirements of significantly increasing data traffic and user demands. In C-RAN, the data processing unit can be centralized and virtualized in data centers and can be shared among distributed base stations. Deep learning (DL) appears to be a feasible approach for facilitating the data processing capability, resource management in the cloud, and predicting dynamic traffic in cellular communication. The convergence of C-RAN and DL is expected to bring new prospects to both interdisciplinary research and industrial applications. In this regard, different approaches have been proposed for DL-based C-RAN in the literature. This article provides a comprehensive survey of the state-of-the-art DL techniques applied in C-RAN. A brief introduction of the C-RAN architecture and DL techniques is given to provide insights into these two emerging technologies. Existing surveys are also discussed to highlight the research gap. The reviewed works are categorized into power consumption optimization, network performance maximization, and QoS maximization based on their optimization objectives. The key ideas of DL applied in the reviewed schemes are also mentioned, and the performance evaluation techniques used in the research are discussed and compared. Lastly, research challenges and open research issues are highlighted to provide future research directions.INDEX TERMS Cloud radio access network, deep learning, evaluation techniques, optimization objectives, performance metrics.

show abstract

“…A preliminary version of this paper was presented at IEEE CCET 2020 [16]. This version is extension of that paper which focuses on EE maximization instead of power minimization.…”

Section: Introductionmentioning

confidence: 99%

“…3. Previous works [16], [31], and [35] rely on the static state-space feature to maximize the certain objective function. In this work, we optimize the objective function by considering the dynamic state-space feature, which is dynamically updated by the movement of UEs at each step.…”

Section: Introductionmentioning

confidence: 99%

Double Deep Q-Network-Based Energy-Efficient Resource Allocation in Cloud Radio Access Network

2021

Self Cite

View full text Add to dashboard Cite

Cloud radio access network (CRAN) has been shown as an effective means to boost network performance. Such gain stems from the intelligent management of remote radio heads (RRHs) in terms of on/off operation mode and power consumption. Most conventional resource allocation (RA) methods, however, optimize the network utility without considering the switching overhead of RRHs in adjacent time intervals. When the network environment becomes time-correlated, mathematical optimization is not directly applicable. In this paper, we aim to optimize the energy efficiency (EE) subject to the constraints on per-RRH transmission power and user data rates. To this end, we formulate the EE problem as a Markov decision process (MDP) and subsequently adopt deep reinforcement learning (DRL) technique to reap the cumulative EE rewards. Our starting point is the deep Q network (DQN), which is a combination of deep learning and Q-learning. In each time slot, DQN configures the status of RRHs yielding the largest Q-value (known as state-action value) prior to solving a power minimization problem for active RRHs. To overcome the Q-value overestimation issue of DQN, we propose a Double DQN (DDQN) framework that obtains optimal reward better than DQN by separating the selected action from the target Q-value generator. Simulation results validate that the DDQN-based RA method is more energy-efficient than the DQN-based RA algorithm and a baseline solution.

show abstract

Double Deep Q-Network for Power Allocation in Cloud Radio Access Network

Cited by 15 publications

References 20 publications

Proactive Power Control and Position Deployment for Drone Small Cells: Joint Supervised and Unsupervised Learning

Proactive Power Control and Position Deployment for Drone Small Cells: Joint Supervised and Unsupervised Learning

A Survey on Applications of Deep Learning in Cloud Radio Access Network

Double Deep Q-Network-Based Energy-Efficient Resource Allocation in Cloud Radio Access Network

Contact Info

Product

Resources

About