LoRaWAN is a media access control (MAC) protocol for wide area networks. It is designed to allow low-powered devices to communicate with Internet-connected applications over long-range wireless connections. The targeted dense deployment will inevitably cause a shortage in radio resources. Hence, autonomous and lightweight radio resource management is crucial to offer ultra-long battery lifetime for LoRa devices. One of the most promising solutions to such a challenge is the use of artificial intelligence. This will enable LoRa devices to use innovative and inherently distributed learning techniques, thus freeing them from draining their limited energy by constantly communicating with a centralized controller.Before proceeding with the deployment of self-managing solutions on top of a LoRaWAN application, it is sensible to conduct simulation-based studies to optimize the design of learningbased algorithms as well as the application under consideration. Unfortunately, a network simulator for such a context is not fully considered or lacks real deployment parameters. In order to address this shortcoming, we have developed a LoRaWAN simulator which aims for resources allocation problem in LoRaWAN network. The Multi-Armed Bandit and its reinforcement learning based algorithm are used to formulate and finding a resource allocation solution. To demonstrate the usefulness of our simulator, extensive simulations were run in a realistic environment taking into account physical phenomenon in LoRaWAN such as the capture effect and inter-spreading factor interference. The simulation results show that the proposed simulator provides a flexible and efficient environment to evaluate various network design parameters and self-management solutions as well as verify the effectiveness of the distributed learning algorithms for resource allocation problems in LoRaWAN.

For a seamless deployment of the Internet of Things (IoT), self-managing solutions are needed to overcome the challenges of IoT, including massively dense networks and careful management of constrained resources in terms of calculation, memory, and battery. Leveraging on artificial intelligence will enable IoT devices to operate autonomously by using inherently distributed learning techniques. Fully distributed resource management will free devices from draining their limited energy by constantly communicating with a centralized controller. The present work is devoted to a specific IoT context, that of LoRaWAN, where devices communicate with the access network via ALOHA-type access and spread spectrum technology. Concurrent transmissions on different spreading factors increase the network capacity. However, the bottleneck is inevitable with the expected massive deployment of LoRa devices. To address this issue, we resort to the popular EXP3 (Exponential Weights for Exploration and Exploitation) algorithm to steer autonomously the decision of LoRa devices towards the least solicited spreading factors. Furthermore, the spreading factor selection is cast as a proportional fair optimization problem used as a benchmark for the learning-based algorithm. Extensive simulations were run in a realistic environment taking into account physical phenomena in LoRaWAN such as the capture effect and inter-spreading factor collision, as well as non-uniform device distribution. In such a realistic setting, we evaluate the performances of the EXP3.S algorithm, an efficient variant of the EXP3 algorithm, and show its relevance against the fair centralized solution and basic heuristics.

Nowadays, with the abundance of diverse air interfaces in the same operating area, advanced Radio Resource Management (RRM) is vital to take advantage of the available system resources. In such a scenario, a mobile user will be able to connect concurrently to different wireless access networks. In this paper, we consider the downlink of a hybrid network with two broadband Radio Access Technologies (RAT): WiMAX [1] and WiFi [2]. Two approaches are proposed to load balance the traffic of every user between the two available RATs: an individual approach where mobile users selfishly strive to improve their performance and a global approach where resource allocation is made in a way to satisfy all mobile users. We devise for the individual approach a fully distributed resource management scheme portrayed as a non-cooperative game. We characterize the Nash equilibriums of the proposed RRM game and put forward a decentralized algorithm based on replicator dynamics to achieve those equilibriums. In the global approach, resources are assigned by the system in order to enhance global performances. For the two approaches, we show that after convergence, each user is connected to a single RAT which avoids costly traffic splitting between available RATs.Key-words: Non-cooperative game theory, non-linear optimisation, WiMAX, WiFi, 4G networks.Approches Individuelle et globale pour la gestion des ressources radio dans un réseau large-bande hybride Résumé : Actuellement, avec la diversification des interfaces radio présentes dans une même zone géographique, il devient vital de mettre en place des techniques avancées de gestion de ressources radio afin de profiter des ressources disponibles. Dans un tel scénario, un utilisateur mobile peut se connecter simultanémentà différents réseaux d'accès sans-fil. Dans ce papier, nous considérons la voie descendante d'un réseau hybride avec deux technologies d'accès radio large-bande disponibles: WiMAX [1] et WiFi [2]. Deux approches sont proposées pouréquilibrer le trafic de chaque utilisateur sur les deux technologies d'accès: une approche individuelle où les mobiles cherchentà améliorer leurs performances d'une manièreégoste et une approche globale où l'allocation de ressources est faite de façonà satisfaire tous les utilisateurs mobiles. Nous introduisons pour l'approche individuelle un schéma de gestion distribuée des ressources formulée en un jeu non-coopératif. Nous caractérisons leséquilibres de Nash du jeu proposé et proposons un algorithme distribué pour atteindre ceséquilibres. Pour l'approche globale, les ressources sont allouées par le système dans le but d'améliorer les performances globales. Pour les deux approches, nous démontrons qu'après convergence, chaque utilisateur se trouve connectéà une seule technologie d'accès sans-fil, ce qui permet d'éviter un partage de trafic coûteux entre les technologies disponibles.

When several radio access technologies (e.g., HSPA, LTE, WiFi and WiMAX) cover the same region, deciding to which one mobiles connect is known as the Radio Access Technology (RAT) selection problem. To reduce network signaling and processing load, decisions are generally delegated to mobile users. Mobile users aim to selfishly maximize their utility. However, as they do not cooperate, their decisions may lead to performance inefficiency. In this paper, to overcome this limitation, we propose a network-assisted approach. The network provides information for the mobiles to make more accurate decisions. By appropriately tuning network information, user decisions are globally expected to meet operator objectives, avoiding undesirable network states. Deriving network information is formulated as a Semi-Markov Decision Process (SMDP), and optimal policies are computed using the Policy Iteration algorithm. Also, and since network parameters may not be easily obtained, a reinforcement learning approach is introduced to derive what to signal to mobiles. The performances of optimal, learning-based, and heuristic policies, such as blocking probability and average throughput, are analyzed. When tuning thresholds are pertinently set, our heuristic achieves performance very close to the optimal solution. Moreover, although it provides lower performance, our learning-based algorithm has the crucial advantage of requiring no prior parameterization. Index Terms-Radio access technology selection, semi-Markov decision process, reinforcement learning, heterogeneous cellular networks. I. INTRODUCTION T HE demand for high-quality and high-capacity radio networks is continuously increasing. It has been reported that global mobile data traffic grew by 81 percent in 2013 [1]. Furthermore, monthly mobile traffic is forecast to surpass 15 exabytes by 2018, nearly 10 times more than in 2013 [1]. Along with this impressive growth, mobile operators are urged to intelligently invest in network infrastructure. They also need to reconsider their flat-rate pricing models [2], seeking positive return-on-investment.

No abstract