Self-Organization in Small Cell Networks: A Reinforcement Learning Approach

Bennis, Mehdi; Perlaza, Samir M.; Blasco, Pol; Han, Zhu; Poor, H. Vincent

doi:10.1109/twc.2013.060513.120959

Cited by 187 publications

(145 citation statements)

References 28 publications

Supporting

Mentioning

139

Contrasting

Order By: Relevance

“…It is capable of learning an unknown environment's statistics as well as of taking actions in the environment so as to maximize the cumulative reward, where the environment itself may be changed by the agent's actions. Reinforcement learning has been widely adopted in communications and networks [37], in control [38], in finance and economics [39], as well as in social science [40]. Specifically, Xiao et.…”

Section: B Spontaneous Credibility Equilibrium: Reinforcement Learningmentioning

confidence: 99%

“…Finally, let us define σ R = 100 − ρ k , which is the percentage of users sharing genuine ratings. In such a case, for each σ R , we can evaluate the corresponding utility of each user by implementing the collaborative filtering algorithm of (37)(38) and the recommendation quality evaluation of (39). Fig.…”

Section: A Reward Function Verification 1) Recommender Systemsmentioning

confidence: 99%

See 1 more Smart Citation

Information Credibility Modeling in Cooperative Networks: Equilibrium and Mechanism Design

Jiang

Han

Ren

et al. 2017

IEEE J. Select. Areas Commun.

Self Cite

View full text Add to dashboard Cite

Abstract-In a cooperative network the user equipment (UE) share information with each other for cooperatively achieving a common goal. However, owing to the concerns of privacy or cost, UEs may be reluctant to share genuine information, which raises the information credibility problem addressed. Diverse techniques have been proposed for enhancing the information credibility in various scenarios. However, there is paucity of information on modeling the UEs' decision making behavior, namely as to whether they are willing/able to share genuine information, even though this directly affects the information credibility across the network. Hence, we propose a game theoretic framework for the associated information credibility modelling by taking into account the users' information sharing strategies and utilities. This framework is investigated under both a homogeneous model and a heterogeneous model. The spontaneous information credibility equilibria of both models are derived and analyzed, including the closed-form analysis of the homogeneous model based on a sophisticated evolutionary game model and on the reinforcement learning based analysis of the heterogeneous model. Moreover, a credit mechanism is designed for encouraging the UEs to share genuine information. Experimental results relying on real-world data traces support our utility function formulation, while our simulation results verify the theoretical analysis and show that all UEs are encouraged by the proposed algorithm to share genuine information with a probability of one, when a credit mechanism is invoked. The proposed modelling techniques may be applied in diverse cooperative networks, including classic wireless networks, vehicular networks, as well as social networks.

show abstract

Section: B Spontaneous Credibility Equilibrium: Reinforcement Learningmentioning

confidence: 99%

Section: A Reward Function Verification 1) Recommender Systemsmentioning

confidence: 99%

Information Credibility Modeling in Cooperative Networks: Equilibrium and Mechanism Design

Jiang

Han

Ren

et al. 2017

IEEE J. Select. Areas Commun.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Among other approaches, including those based on reinforcement learning, maximum-entropy reinforcement learning, smoothed best-response or fictitious play, it is important to highlight the contributions in [3], [7], [8], [18]- [23]. The main drawbacks of these contributions can be summarized in five points: (i) The converging point is a probability distribution over the set of all available channels and power allocations policies [21], [22], [30], [31]. Therefore, the optimization is often on the expectation of the performance metric and the optimality is often claimed in the asymptotic regime.…”

Section: A State Of the Artmentioning

confidence: 99%

Self-Organization in Decentralized Networks: A Trial and Error Learning Approach

Rose

Perlaza

Martret

et al. 2014

IEEE Trans. Wireless Commun.

Self Cite

View full text Add to dashboard Cite

Abstract-In this paper, the problem of channel selection and power control is jointly analyzed in the context of multiplechannel clustered ad-hoc networks, i.e., decentralized networks in which radio devices are arranged into groups (clusters) and each cluster is managed by a central controller (CC). This problem is modeled by game in normal form in which the corresponding utility functions are designed for making some of the Nash equilibria (NE) to coincide with the solutions to a global network optimization problem. In order to ensure that the network operates in the equilibria that are globally optimal, a learning algorithm based on the paradigm of trial and error learning is proposed. These results are presented in the most general form and therefore, they can also be seen as a framework for designing both games and learning algorithms with which decentralized networks can operate at global optimal points using only their available local knowledge. The pertinence of the game design and the learning algorithm are highlighted using specific scenarios in decentralized clustered ad hoc networks. Numerical results confirm the relevance of using appropriate utility functions and trial and error learning for enhancing the performance of decentralized networks.

show abstract

“…They proposed a Markov approximation framework to study the convergence in probability. In [2], the authors studied a reinforcement-learning based framework for interference management in small cells networks and proposed self-organizing strategies for interference management in closed-access small cell networks with minimum information required to learn an equilibrium. In [3], the authors employed a semi-Markov decision process to study the admission control problem and design a power control game to reduce energy consumption.…”

Section: Introductionmentioning

confidence: 99%

SDN-Based Self-Organizing Energy Efficient Downlink/Uplink Scheduling in Heterogeneous Cellular Networks

Moon

Kazmi

et al. 2017

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

Seungil MOON†a) , Thant Zin OO †b) , S. M. Ahsan KAZMI †c) , Bang Ju PARK † †d) , Nonmembers, and Choong Seon HONG †e) , Member SUMMARYThe increase in network access devices and demand for high quality of service (QoS) by the users have led to insufficient capacity for the network operators. Moreover, the existing control equipment and mechanisms are not flexible and agile enough for the dynamically changing environment of heterogeneous cellular networks (HetNets). This nonagile control plane is hard to scale with ever increasing traffic demand and has become the performance bottleneck. Furthermore, the new HetNet architecture requires tight coordination and cooperation for the densely deployed small cell base stations, particularly for interference mitigation and dynamic frequency reuse and sharing. These issues further complicate the existing control plane and can cause serious inefficiencies in terms of users' quality of experience and network performance. This article presents an SDN control framework for energy efficient downlink/uplink scheduling in HetNets. The framework decouples the control plane from data plane by means of a logically centralized controller with distributed agents implemented in separate entities of the network (users and base stations). The scheduling problem consists of three sub-problems: (i) user association, (ii) power control, (iii) resource allocation and (iv) interference mitigation. Moreover, these sub-problems are coupled and must be solved simultaneously. We formulate the DL/UL scheduling in HetNet as an optimization problem and use the Markov approximation framework to propose a distributed economical algorithm. Then, we divide the algorithm into three sub-routines for (i) user association, (ii) power control, (iii) resource allocation and (iv) interference mitigation. These sub-routines are then implemented on different agents of the SDN framework. We run extensive simulation to validate our proposal and finally, present the performance analysis.

show abstract

Self-Organization in Small Cell Networks: A Reinforcement Learning Approach

Cited by 187 publications

References 28 publications

Information Credibility Modeling in Cooperative Networks: Equilibrium and Mechanism Design

Information Credibility Modeling in Cooperative Networks: Equilibrium and Mechanism Design

Self-Organization in Decentralized Networks: A Trial and Error Learning Approach

SDN-Based Self-Organizing Energy Efficient Downlink/Uplink Scheduling in Heterogeneous Cellular Networks

Contact Info

Product

Resources

About