ALOHA and Q-Learning based medium access control for Wireless Sensor Networks

Chu, Yi; Mitchell, Paul; Grace, David

doi:10.1109/iswcs.2012.6328420

Cited by 61 publications

(54 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In WSNs, a number of sensors cooperate to efficiently transfer data. Therefore, designing MAC protocols for WSNs poses different challenges from typical wireless networks, as well as energy consumption and latency [112]. Also, the duty cycle (i.e., fraction of time that a sensor node is active) of the node has to be controlled to conserve energy.…”

Section: E Medium Access Control (Mac)mentioning

confidence: 99%

“…Similarly, Chu et al [112] integrated slotted ALOHA and Q-Learning algorithms to introduce a new MAC protocol for WSNs, called "ALOHA and Q-Learning based MAC with Informed Receiving" (ALOHA-QIR). ALOHA-QIR inherits the features of both ALOHA and Q-Learning to achieve the benefits of simple design, low resource requirements and low collision probability.…”

Section: E Medium Access Control (Mac)mentioning

confidence: 99%

See 1 more Smart Citation

Machine Learning in Wireless Sensor Networks: Algorithms, Strategies, and Applications

Alsheikh

Lin

Niyato

et al. 2014

IEEE Commun. Surv. Tutorials

811

406

View full text Add to dashboard Cite

Abstract-Wireless sensor networks monitor dynamic environments that change rapidly over time. This dynamic behavior is either caused by external factors or initiated by the system designers themselves. To adapt to such conditions, sensor networks often adopt machine learning techniques to eliminate the need for unnecessary redesign. Machine learning also inspires many practical solutions that maximize resource utilization and prolong the lifespan of the network. In this paper, we present an extensive literature review over the period 2002-2013 of machine learning methods that were used to address common issues in wireless sensor networks (WSNs). The advantages and disadvantages of each proposed algorithm are evaluated against the corresponding problem. We also provide a comparative guide to aid WSN designers in developing suitable machine learning solutions for their specific application challenges.

show abstract

Section: E Medium Access Control (Mac)mentioning

confidence: 99%

Section: E Medium Access Control (Mac)mentioning

confidence: 99%

Machine Learning in Wireless Sensor Networks: Algorithms, Strategies, and Applications

Alsheikh

Lin

Niyato

et al. 2014

IEEE Commun. Surv. Tutorials

811

406

View full text Add to dashboard Cite

show abstract

“…We introduce a separate DQN agent for each output variable in A t defined as action A t k selected by the kth agent, where each kth agent is responsible to update the value Q(S t , A t k ; θ k ) of action A t k in shared 6 The action aggregation has been rarely evaluated, but the same idea, namely, state aggregation has been well studied, which is a basic function approximation approach [31]. 7 The structures of value function approximator can also be specifically designed for RL agents with sub-tasks of significantly different complexity. However, there is no such requirement in our problem, so it will not be considered.…”

Section: B Cooperative Multi-agent Learning Approachmentioning

confidence: 99%

“…In order to consider more complex and practical formulations, Reinforcement Learning (RL) emerges as a natural solution given its capability in interacting with the practical environment and feedback in the form of the number of successful and unsuccessful transmissions per TTI. Distributed RL based on tabular Q-learning (tabular-Q) has been proposed in [6][7][8][9]. In [6][7][8], the authors studied distributed tabular-Q in slotted-Aloha networks, where each device learns how to avoid collisions by finding a proper time slot to transmit packets.…”

Section: Introductionmentioning

confidence: 99%

Reinforcement Learning for Real-Time Optimization in NB-IoT Networks

Jiang

Deng

Nallanathan

et al. 2019

IEEE J. Select. Areas Commun.

View full text Add to dashboard Cite

NarrowBand-Internet of Things (NB-IoT) is an emerging cellular-based technology that offers a range of flexible configurations for massive IoT radio access from groups of devices with heterogeneous requirements. A configuration specifies the amount of radio resource allocated to each group of devices for random access and for data transmission. Assuming no knowledge of the traffic statistics, there exists an important challenge in "how to determine the configuration that maximizes the long-term average number of served IoT devices at each Transmission Time Interval (TTI) in an online fashion". Given the complexity of searching for optimal configuration, we first develop real-time configuration selection based on the tabular Q-learning (tabular-Q), the Linear Approximation based Q-learning (LA-Q), and the Deep Neural Network based Q-learning (DQN) in the single-parameter single-group scenario. Our results show that the proposed reinforcement learning based approaches considerably outperform the conventional heuristic approaches based on load estimation (LE-URC) in terms of the number of served IoT devices. This result also indicates that LA-Q and DQN can be good alternatives for tabular-Q to achieve almost the same performance with much less training time. We further advance LA-Q and DQN via Actions Aggregation (AA-LA-Q and AA-DQN) and via Cooperative Multi-Agent learning (CMA-DQN) for the multi-parameter multi-group scenario, thereby solve the problem that Q-learning agents do not converge in high-dimensional configurations. In this scenario, the superiority of the proposed Q-learning approaches over the conventional LE-URC approach significantly improves with the increase of configuration dimensions, and the CMA-DQN approach outperforms the other approaches in both throughput and training efficiency. Partnership Project (3GPP) partners have standardized a new radio access technology, namely NarrowBand-IoT (NB-IoT) [1]. NB-IoT is expected to provide reliable wireless access for IoT devices with various

show abstract

“…RL-based protocols significantly reduce the energy consumption due to both idle listening and overhearing in the context of duty cycling. ALOHA and Q-Learning have been integrated to establish a new MAC protocol, namely ALOHA-Q [3]. ALOHA-based techniques are important for certain categories of Wireless Personal Networks (WPNs) and…”

Section: Reinforcement Learning (Rl) Has Been Recently Applied To Desmentioning

confidence: 99%

Use of Q-learning approaches for practical medium access control in wireless sensor networks

Koşunalp

Chu

Mitchell

et al. 2016

Engineering Applications of Artificial Intelligence

Self Cite

View full text Add to dashboard Cite

This paper studies the potential of a novel approach to ensure more efficient and intelligent assignment of capacity through medium access control (MAC) in practical wireless sensor networks. Q-Learning is employed as an intelligent transmission strategy. We review the existing MAC protocols in the context of Q-learning. A recently-proposed, ALOHA and Q-Learning based MAC scheme, ALOHA-Q, is considered which improves the channel performance significantly with a key benefit of simplicity. Practical implementation issues of ALOHA-Q are studied. We demonstrate the performance of the ALOHA-Q through extensive simulations and evaluations in various testbeds. A new exploration/exploitation method is proposed to strengthen the merits of the ALOHA-Q against dynamic the channel and environment conditions.

show abstract

ALOHA and Q-Learning based medium access control for Wireless Sensor Networks

Cited by 61 publications

References 12 publications

Machine Learning in Wireless Sensor Networks: Algorithms, Strategies, and Applications

Machine Learning in Wireless Sensor Networks: Algorithms, Strategies, and Applications

Reinforcement Learning for Real-Time Optimization in NB-IoT Networks

Use of Q-learning approaches for practical medium access control in wireless sensor networks

Contact Info

Product

Resources

About