Q-Learning-Based Power Control for LTE Enterprise Femtocell Networks

Gao, Zhibin; Wen, Bin; Huang, Lianfen; Chen, Canbin; Su, Ziwen

doi:10.1109/jsyst.2016.2535461

Cited by 28 publications

(19 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In fact, RL operates by applying the experience that it has gained through interacting with the network [18]. RL methods have been applied in the field of wireless communications in areas such as resource management [19]- [24], energy harvesting [25], and opportunistic spectrum access [26], [27]. A comprehensive review of RL applications in wireless communications can be found in [28].…”

Section: A Related Workmentioning

confidence: 99%

“…In this regard, the works in [19]- [24] have proposed different reward functions to optimize power allocation between femtocell base stations (FBSs). The method in [19] uses independent Q-learning in a cognitive radio system to set the transmit power of secondary BSs in a digital television system.…”

Section: A Related Workmentioning

confidence: 99%

“…In [23], the authors attempt to improve the transmission rate of cell-edge users while keeping the fairness between the macrocell and the femtocell users by applying a round robin approach. The work in [24] minimizes power usage in a Long Term Evolution (LTE) enterprise femtocell network by applying an exponential reward function without the requirement to achieve fairness amongst the femtocells in the network.…”

Section: A Related Workmentioning

confidence: 99%

“…Constraints (3c) and (3d) ensure that the minimum SINR requirement is satisfied for the MUE and the FUEs. The addition of constraint (3d) to the optimization problem is one of the differences between the proposed approach in this paper and that of [19]- [24].…”

Section: Problem Formulationmentioning

confidence: 99%

“…In the prior works [19]- [21], [23], [24], a constant learning rate was used for Q-learning to solve the required optimization problems. However, according to [38], in finite number of iterations, the performance of Q-learning can be improved by applying a decaying learning rate.…”

Section: A Q-learning Based Distributed Power Allocation (Q-dpa)mentioning

confidence: 99%

See 4 more Smart Citations

Reinforcement Learning for Self Organization and Power Control of Two-Tier Heterogeneous Networks

Amiri

Almasi

Andrews

et al. 2019

IEEE Trans. Wireless Commun.

View full text Add to dashboard Cite

Self-organizing networks (SONs) can help manage the severe interference in dense heterogeneous networks (HetNets). Given their need to automatically configure power and other settings, machine learning is a promising tool for data-driven decision making in SONs. In this paper, a HetNet is modeled as a dense two-tier network with conventional macrocells overlaid with denser small cells (e.g. femto or pico cells). First, a distributed framework based on multi-agent Markov decision process is proposed that models the power optimization problem in the network. Second, we present a systematic approach for designing a reward function based on the optimization problem. Third, we introduce Q-learning based distributed power allocation algorithm (Q-DPA) as a self-organizing mechanism that enables ongoing transmit power adaptation as new small cells are added to the network. Further, the sample complexity of the Q-DPA algorithm to achieve -optimality with high probability is provided. We demonstrate, at density of several thousands femtocells per km 2 , the required quality of service of a macrocell user can be maintained via the proper selection of independent or cooperative learning and appropriate Markov state models.

show abstract

Section: A Related Workmentioning

confidence: 99%

Section: A Related Workmentioning

confidence: 99%

Section: A Related Workmentioning

confidence: 99%

Section: Problem Formulationmentioning

confidence: 99%

Section: A Q-learning Based Distributed Power Allocation (Q-dpa)mentioning

confidence: 99%

See 3 more Smart Citations

Reinforcement Learning for Self Organization and Power Control of Two-Tier Heterogeneous Networks

Amiri

Almasi

Andrews

et al. 2019

IEEE Trans. Wireless Commun.

View full text Add to dashboard Cite

show abstract

Q-Learning Based Joint Allocation of Fronthaul and Radio Resources in Multiwavelength-Enabled C-RAN

Mikaeil¹,

Hu²

2020

Optical Network Design and Modeling

View full text Add to dashboard Cite

Multi-wavelengths passive optical networks (PONs) such as wavelength division multiplexing (WDM) and time wavelength division multiplexing (TWDM) PONs are outstanding solutions for providing a sufficient bandwidth for mobile front-haul to support C-RAN architecture in 5G mobile network. In this paper a joint allocation framework for multi-wavelength PONs mobile front-haul and C-RAN air interface uplink resources is proposed. From the principle that uplink resource allocation in mobile networks (e.g. 4G and 5G) is an NP-hard optimization problem, this paper contributes with a novel method for uplink scheduling based on a reinforcement learning (RL) algorithm known as Q-Learning. The performance of the algorithm is evaluated with numerical simulations and compared with some other relevant work from the literature such as genetic algorithm (GA) and tabu search (TS). The simulation results show that the new algorithm achieves faster convergence, higher throughput, and minimum scheduling time compared to the two other algorithms. The results also show that RL-based dynamic allocation of front-haul transport block capacity based on actual radio resource block size can greatly reduce front-haul capacity requirement and minimize total end to end uplink scheduling latency.

show abstract

QoS-aware cooperative power control and resource allocation scheme in LTE femtocell networks

Wang

Kuo

Chu

2017

Computer Communications

View full text Add to dashboard Cite

Q-Learning-Based Power Control for LTE Enterprise Femtocell Networks

Cited by 28 publications

References 20 publications

Reinforcement Learning for Self Organization and Power Control of Two-Tier Heterogeneous Networks

Reinforcement Learning for Self Organization and Power Control of Two-Tier Heterogeneous Networks

Q-Learning Based Joint Allocation of Fronthaul and Radio Resources in Multiwavelength-Enabled C-RAN

QoS-aware cooperative power control and resource allocation scheme in LTE femtocell networks

Contact Info

Product

Resources

About