Wireless Access Control in Edge-Aided Disaster Response: A Deep Reinforcement Learning-Based Approach

Zhou, Hang; Wang, Xiaoyan; Umehira, Masahiro; Chen, Xianfu; Wu, Celimuge; Ji, Yusheng

doi:10.1109/access.2021.3067662

Cited by 18 publications

(4 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In order to maximize the expected achievable rate of UE in ultra-dense networks, the authors [ 51 ] developed a matching game algorithm, where mobility-aware user association was considered by minimizing the handovers number. The authors [ 52 ] deployed the DRL algorithm to estimate the transmit timing, routing as well as power allocation for UEs from MDRU deployed in disaster areas where UE mobility, channel states, and energy harvesting were considered.…”

Section: Related Workmentioning

confidence: 99%

Mobility-Aware Resource Allocation in IoRT Network for Post-Disaster Communications with Parameterized Reinforcement Learning

Kabir

Tham

Chang

et al. 2023

Sensors

View full text Add to dashboard Cite

Natural disasters, including earthquakes, floods, landslides, tsunamis, wildfires, and hurricanes, have become more common in recent years due to rapid climate change. For Post-Disaster Management (PDM), authorities deploy various types of user equipment (UE) for the search and rescue operation, for example, search and rescue robots, drones, medical robots, smartphones, etc., via the Internet of Robotic Things (IoRT) supported by cellular 4G/LTE/5G and beyond or other wireless technologies. For uninterrupted communication services, movable and deployable resource units (MDRUs) have been utilized where the base stations are damaged due to the disaster. In addition, power optimization of the networks by satisfying the quality of service (QoS) of each UE is a crucial challenge because of the electricity crisis after the disaster. In order to optimize the energy efficiency, UE throughput, and serving cell (SC) throughput by considering the stationary as well as movable UE without knowing the environmental priori knowledge in MDRUs aided two-tier heterogeneous networks (HetsNets) of IoRT, the optimization problem has been formulated based on emitting power allocation and user association combinedly in this article. This optimization problem is nonconvex and NP-hard where parameterized (discrete: user association and continuous: power allocation) action space is deployed. The new model-free hybrid action space-based algorithm called multi-pass deep Q network (MP-DQN) is developed to optimize this complex problem. Simulations results demonstrate that the proposed MP-DQN outperforms the parameterized deep Q network (P-DQN) approach, which is well known for solving parameterized action space, DQN, as well as traditional algorithms in terms of reward, average energy efficiency, UE throughput, and SC throughput for motionless as well as moveable UE.

show abstract

Section: Related Workmentioning

confidence: 99%

Mobility-Aware Resource Allocation in IoRT Network for Post-Disaster Communications with Parameterized Reinforcement Learning

Kabir

Tham

Chang

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…38 It describes a sequence of exploration–exploitation decision-making processes. 39,40 The MAB model is mainly composed of decision makers, arms, and rewards. 41 In each round, the decision maker selects an arm, and the selected arm will generate a reward.…”

Section: Delay-reliability-aware Mqtt Qos Level Selection In Elotmentioning

confidence: 99%

Delay-reliability-aware protocol adaption and quality of service guarantee for message queuing telemetry transport-empowered electric Internet of things

Zhang

Zhao

et al. 2022

International Journal of Distributed Sensor Networks

View full text Add to dashboard Cite

Message queuing telemetry transport has emerged as a promising communication protocol for resource-constrained electric Internet of things due to high bandwidth utilization, simple implementation, and various quality of service levels. Enabled by message queuing telemetry transport, electric Internet of things gateways adopt dynamic protocol adaptation, conversion, and quality of service level selection to realize bidirectional communication with massive devices and platforms based on heterogeneous communication protocols. However, protocol adaptation and quality of service guarantee in message queuing telemetry transport-empowered electric Internet of things still faces several challenges, such as unified communication architecture, differentiated quality of service requirements, lack of quality of service metric models, and incomplete information. In this paper, we first establish a unified communication architecture for message queuing telemetry transport-empowered electric Internet of things for adaptation and conversion of heterogeneous protocols. Second, we formulate the quality of service level selection optimization problem to minimize the weighted sum of packet-loss ratio and delay. Then, a delay-reliability-aware message queuing telemetry transport quality of service level selection algorithm based on upper confidence bound is proposed to learn the optimal quality of service level through dynamically interacting with the environment. Compared with single and fixed quality of service level selection strategies, delay-reliability-aware message queuing telemetry transport quality of service level selection can effectively reduce the weighted sum of delay and packet-loss ratio and satisfy the differentiated quality of service requirements of electric Internet of things.

show abstract

“…The proposed scheme is based on a typical reinforcement learning algorithm, i.e., Qlearning [25,26]. Reinforcement learning is one of the three basic machine learning paradigms, by which the agent learns the optimal behavior through repeated interactions with the environment in discrete time steps.…”

Section: Proposed Reinforcement Learning-basedmentioning

confidence: 99%

Reinforcement Learning for Joint Channel/Subframe Selection of LTE in the Unlicensed Spectrum

Kishimoto

Wang

Umehira

2021

Wireless Communications and Mobile Computing

Self Cite

View full text Add to dashboard Cite

In recent years, to cope with the rapid growth in mobile data traffic, increasing the capacity of cellular networks is receiving more and more attention. To this end, offloading the current LTE-advanced or 5G system’s data traffic from licensed spectrum to the unlicensed spectrum that is used by WiFi systems, i.e., LTE-Licensed-Assisted-Access (LTE-LAA), has been extensively investigated. In the current LTE-LAA system, a Listen-Before-Talk (LBT) approach is implemented, which requires the LTE user also perform carrier sense before the transmission. However, fair LTE-WiFi coexistence is still hard to guarantee due to their unbalanced frame sizes and traffic loads. In the LTE-LAA system, the optimal channel selection and subframe number adjustment are the keys to realize efficient spectrum utilization and fair system coexistence. To this end, in this paper, we propose a reinforcement learning-based joint channel/subframe selection scheme for LTE-LAA. The proposed approach is implemented at the LTE access points with zero knowledge of the WiFi systems. The results of extensive simulations verify that the proposed approach can significantly improve the fairness and packet loss rate compared with baseline schemes.

show abstract

Wireless Access Control in Edge-Aided Disaster Response: A Deep Reinforcement Learning-Based Approach

Cited by 18 publications

References 30 publications

Mobility-Aware Resource Allocation in IoRT Network for Post-Disaster Communications with Parameterized Reinforcement Learning

Mobility-Aware Resource Allocation in IoRT Network for Post-Disaster Communications with Parameterized Reinforcement Learning

Delay-reliability-aware protocol adaption and quality of service guarantee for message queuing telemetry transport-empowered electric Internet of things

Reinforcement Learning for Joint Channel/Subframe Selection of LTE in the Unlicensed Spectrum

Contact Info

Product

Resources

About