This thesis investigates the application of Reinforcement Learning (RL) on Medium Access Control (MAC) for Wireless Sensor Networks (WSNs). RL is applied as an intelligent slot selection strategy to Framed ALOHA, along with analytical and experimental performance evaluation. Informed Receiving (IR) and ping packets are applied to multi-hop WSNs to avoid idle listening and overhearing, thereby further improving the energy efficiency.The low computational complexity and signalling overheads of the ALOHA schemes meet the design requirement of energy constraint WSNs, but suffer collisions from the random access strategy. RL is applied to solve this problem and to achieve perfect scheduling. Results show that the RL scheme achieves over 0.9 Erlangs maximum throughput in single-hop networks. For multi-hop WSNs, IR and ping packets are applied to appropriately switch the relay nodes between active and sleep state, to reserve as much energy as possible while ensuring no information loss.The RL algorithms require certain time to converge to steady state to achieve the optimum performance. The convergence behaviour is investigated in this thesis. A Markov model is proposed to describe a learning process, and the model produces the proof of the convergence of the learning process and the estimated convergence time. The channel performance before convergence is also evaluated.Contents

No abstract

This paper studies the potential of a novel approach to ensure more efficient and intelligent assignment of capacity through medium access control (MAC) in practical wireless sensor networks. Q-Learning is employed as an intelligent transmission strategy. We review the existing MAC protocols in the context of Q-learning. A recently-proposed, ALOHA and Q-Learning based MAC scheme, ALOHA-Q, is considered which improves the channel performance significantly with a key benefit of simplicity. Practical implementation issues of ALOHA-Q are studied. We demonstrate the performance of the ALOHA-Q through extensive simulations and evaluations in various testbeds. A new exploration/exploitation method is proposed to strengthen the merits of the ALOHA-Q against dynamic the channel and environment conditions.

In this paper, an ALOHA based Medium Access Control (MAC) protocol (RL-ALOHA with Informed Receiving) is proposed for multi-hop Wireless Sensor Networks (WSNs), which overcomes the traditional problems of low throughput, while exploiting their advantages of simplicity, low computational complexity and overheads. Reinforcement Learning (RL) is implemented as an intelligent slot assignment strategy in order to avoid collisions with minimal additional overheads. To improve the energy efficiency, Informed Receiving (IR) and ping packets are applied to avoid idle listening and overhearing. The simulation results show that this approach significantly increases the energy efficiency, achieves over twice throughput of Slotted ALOHA and reduces the end-to-end delay.

With the fast growing number of wireless devices and demand of user data, the backhaul load becomes a bottleneck in wireless networks. Physical‐layer network coding (PNC) allows access points to relay‐compressed, network‐coded user data, therefore reducing the backhaul traffic. In this paper, an implementation of uplink network‐coded modulation (NetCoM) with PNC is presented. A five‐node prototype NetCoM system is established using Universal Software Radio Peripherals, and a practical PNC scheme designed for binary systems is utilized. An orthogonal frequency‐division multiplexing waveform implementation and the practical challenges (eg, device synchronization and clock drift) of applying orthogonal frequency‐division multiplexing to NetCoM are discussed. To the best of our knowledge, this is the first PNC implementation in an uplink scenario in radio access networks, and our prototype provides an industrially applicable implementation of the proposed NetCoM with PNC approach.