Cognition and Docition in OFDMA-Based Femtocell Networks

Galindo-Serrano, Ana; Giupponi, Lorenza; Döhler, Mischa

doi:10.1109/glocom.2010.5683552

Cited by 45 publications

(41 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The contributions of this paper in light of the previous work can be summarized as follows: Establishing a proof for the convergence of the CL paradigm when used in one of our proposed algorithms, that is, the Femto‐based distributed power control using Q‐learning (FBDPC‐Q) algorithm. Extending the path‐loss channel model used in to include a more realistic and generalized channel model, namely, Rayleigh fading, where we show that (1) the gain of using CL for enhancing the aggregate femtocell capacity is more evident compared with IL under the fading model and (2) the reaction of the CL paradigm to the wireless network dynamics (i.e., fading, random activity of femtocells, and density of femtocells) is more robust compared with such reaction for IL. Characterizing the situations in which CL may not outperform IL and propose the weighted CL (WCL) paradigm in order to enhance the performance in such situations. This is attributed to the observation that in the CL paradigm, all of the cooperating femtocells converge toward using the same allocated power, which in some situations, degrades the CL performance. Comparing our proposed schemes in IL, CL, and WCL paradigms to the idea of docitive femtocells presented in . …”

Section: Introductionmentioning

confidence: 88%

Cooperative Q‐learning techniques for distributed online power allocation in femtocell networks

Saad

Mohamed

ElBatt

2014

Wireless Communications

View full text Add to dashboard Cite

In this paper, we address the problem of distributed interference management of femtocells that share the same frequency band with macrocells using distributed multi‐agent Q‐learning. We formulate and solve two problems representing two different Q‐learning algorithms, namely, femto‐based distributed and sub‐carrier‐based distributed power controls using Q‐learning (FBDPC‐Q and SBDPC‐Q). FBDPC‐Q is a multi‐agent algorithm that works on a global basis, for example, deals with the aggregate macrocell and femtocell capacities. Its complexity increases exponentially with the number of sub‐carriers in the system. Also, it does not take into consideration the sub‐carrier macrocell capacity as a constraint. To overcome these problems, SBDPC‐Q is proposed, which is a multi‐agent algorithm that works on a sub‐carrier basis, for example, sub‐carrier macrocell and femtocell capacities. Each of FBDPC‐Q and SBDPC‐Q works in three different learning paradigms: independent (IL), cooperative (CL), and weighted cooperative (WCL). IL is considered the simplest form for applying Q‐learning in multi‐agent scenarios, where all the femtocells learn independently. CL and WCL are the proposed schemes in which femtocells share partial information during the learning process in order to strike a balance between practical relevance and performance. We prove the convergence of the CL paradigm when used in the FBDPC‐Q algorithm. We show via simulations that the CL paradigm outperforms the IL paradigm in terms of the aggregate femtocell capacity, especially in networks with large number of femtocells and large number of power levels. In addition, we propose WCL to address the CL limitations. Finally, we evaluate the robustness and scalability of both FBDPC‐Q and SBDPC‐Q, against several typical dynamics of plausible wireless scenarios (fading, path loss, random activity of femtocells, etc.). We show that the CL paradigm is the most scalable to large number of femtocells and robust to the network dynamics compared with the IL and WCL paradigms. Copyright © 2014 John Wiley & Sons, Ltd.

show abstract

Section: Introductionmentioning

confidence: 88%

Cooperative Q‐learning techniques for distributed online power allocation in femtocell networks

Saad

Mohamed

ElBatt

2014

Wireless Communications

View full text Add to dashboard Cite

show abstract

“…In the absense of such knowledge of system dynamics, learning based schemes such as Q-learning may be used [3]. For example, Q-learning based approaches are used in [5], [6] for problems of delay-constrained wireless transmission scheduling and [7] studies distributed Q-learning for interference control in multiuser cognitive femtocell networks. Q-learning algorithms are general solutions for MDPs that involve learning over time.…”

Section: Introductionmentioning

confidence: 99%

Opportunistic cooperation in cognitive radio networks

Urgaonkar

Neely

2012

2012 Fourth International Conference on Communication Systems and Networks (COMSNETS 2012)

View full text Add to dashboard Cite

We investigate opportunistic cooperation between secondary (unlicensed) users and primary (licensed) users in cognitive radio networks. We consider two models for such cooperation. In the first model, called the Cooperative Relay Model, a secondary user cannot transmit its own data concurrently with a primary user. However, it can employ cooperative relaying in order to improve the effective transmission rate of the primary user. In the second model, called the Interference Model, a secondary user is allowed to transmit its data concurrently with a primary user. However, the secondary user can "cooperate" by deferring its transmissions when the primary user is busy. In both models, the secondary users must make intelligent cooperation decisions as they seek to maximize their own throughput subject to average power constraints. The decision options are different during idle and busy periods of the primary user, and the decisions in turn influence the durations of these periods according to a controllable infinite state Markov chain. Such problems can be formulated as constrained Markov decision problems, and conventional solution techniques require either extensive knowledge of the system dynamics or learning based approaches that suffer from large convergence times. However, using a generalized Lyapunov optimization technique, we design a novel greedy and online control algorithm that overcomes these challenges. Remarkably, this algorithm does not require any knowledge of the network arrival rates and is provably optimal.

show abstract

“…With this valuable knowledge at hand, a newcomer CR is able to reduce the exploration phase and increase its performance. In [11], docition is applied in the context of femtocell networks. Said femtocells use Q-learning, an algorithm coming from reinforcement learning, in order to allocate the optimal power and frequency to transmit.…”

Section: Docitive Net Work Smentioning

confidence: 99%