Design and Implementation of a Simulation System Based on Deep Q-Network for Mobile Actor Node Control in Wireless Sensor and Actor Networks

Oda, Tetsuya; Obukata, Ryoichiro; Ikeda, Makoto; Barolli, Leonard; Takizawa, Makoto

doi:10.1109/waina.2017.67

Cited by 42 publications

(5 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The authors in [157] leverage the DQL framework for sensing and control problems in a Wireless Sensor and Actor Network (WSAN), which is a group of wireless devices with the ability to sense events and to perform actions based on the sensed data shared by all sensors. The system state includes processing power, mobility abilities, and functionalities of the actors and sensors.…”

Section: Power Control and Data Collectionmentioning

confidence: 99%

Applications of Deep Reinforcement Learning in Communications and Networking: A Survey

Luong

Hoang

Gong

et al. 2019

IEEE Commun. Surv. Tutorials

1,487

481

View full text Add to dashboard Cite

This paper presents a comprehensive literature review on applications of deep reinforcement learning in communications and networking. Modern networks, e.g., Internet of Things (IoT) and Unmanned Aerial Vehicle (UAV) networks, become more decentralized and autonomous. In such networks, network entities need to make decisions locally to maximize the network performance under uncertainty of network environment. Reinforcement learning has been efficiently used to enable the network entities to obtain the optimal policy including, e.g., decisions or actions, given their states when the state and action spaces are small. However, in complex and large-scale networks, the state and action spaces are usually large, and the reinforcement learning may not be able to find the optimal policy in reasonable time. Therefore, deep reinforcement learning, a combination of reinforcement learning with deep learning, has been developed to overcome the shortcomings. In this survey, we first give a tutorial of deep reinforcement learning from fundamental concepts to advanced models. Then, we review deep reinforcement learning approaches proposed to address emerging issues in communications and networking. The issues include dynamic network access, data rate control, wireless caching, data offloading, network security, and connectivity preservation which are all important to next generation networks such as 5G and beyond. Furthermore, we present applications of deep reinforcement learning for traffic routing, resource sharing, and data collection. Finally, we highlight important challenges, open issues, and future research directions of applying deep reinforcement learning.

show abstract

Section: Power Control and Data Collectionmentioning

confidence: 99%

Applications of Deep Reinforcement Learning in Communications and Networking: A Survey

Luong

Hoang

Gong

et al. 2019

IEEE Commun. Surv. Tutorials

1,487

481

View full text Add to dashboard Cite

show abstract

“…2) Wireless Sensor and Actuator Networks: Wireless sensor and actuator networks (WSANs), e.g., ISA SP100.11a and WirelessHART, have special devices known as network managers which perform tasks such as admission control of devices, definition of routes, and allocation of communication resources. The authors in [115] present the design and implementation of a simulation system based on DQN for mobile actor node control in a WSAN. In [116], a global routing agent with Q-Learning is proposed for weight adjustment of the state-of-the-art routing algorithm, aiming at achieving a balance between the overall delay and the lifetime of the network.…”

Section: Aiot Network Layer -Iot Communication Networkmentioning

confidence: 99%

Deep Reinforcement Learning for Autonomous Internet of Things: Model, Applications and Challenges

Lei

Tan

Zheng

et al. 2020

IEEE Commun. Surv. Tutorials

226

View full text Add to dashboard Cite

The Internet of Things (IoT) extends the Internet connectivity into billions of IoT devices around the world, which collect and share information to reflect the status of physical world. The Autonomous Control System (ACS), on the other hand, performs control functions on the physical systems without external intervention over an extended period of time. The integration of IoT and ACS results in a new concept -autonomous IoT (AIoT). The sensors collect information on the system status, based on which intelligent agents in IoT devices as well as Edge/Fog/Cloud servers make control decisions for the actuators to react. In order to achieve autonomy, a promising method is for the intelligent agents to leverage the techniques in the field of artificial intelligence, especially reinforcement learning (RL) and deep reinforcement learning (DRL) for decision making. In this paper, we first provide comprehensive survey of the state-of-art research, and then propose a general model for the applications of RL/DRL in AIoT. Finally, the challenges and open issues for future research are identified.

show abstract

“…In addition, few other miscellaneous applications of RL related to communications and networking include user localization ( [209], [72], [210]); direction of arrival estimation [211]; data collection ( [212], [213], [214], [215], [216]); power control ( [217], [218], [219]); signal detection [220] and traffic engineering and routing ( [220], [221], [222], [223], [224], [225], [226], [227]).…”

Section: Communication and Networkingmentioning

confidence: 99%

“…Deep Q-Networks (DQN)algorithms are most frequently adopted in AIOT systems in recent years. A few applications of DQN and Double DQN (DDQN) in IoT communications systems are: [315], [316], [317], [318]and [319]; in IoT Cloud/Fog/Edge computing are: [320], [321], [322], [323] and [324]; in autonomous IoT robotics are: [325], [326], [327], [328] and [329] ; in IoT smart vehicles are: [330], [331] and [299] and in smart grids are: [332], [333], [334] and [335] respectively.…”

Section: E Autonomous Iotmentioning

confidence: 99%

A Gentle Introduction to Reinforcement Learning and its Application in Different Fields

Naeem

Rizvi

Coronato³

2020

IEEE Access

146

View full text Add to dashboard Cite

Due to the recent progress in Deep Neural Networks, Reinforcement Learning (RL) has become one of the most important and useful technology. It is a learning method where a software agent interacts with an unknown environment, selects actions, and progressively discovers the environment dynamics. RL has been effectively applied in many important areas of real life. This article intends to provide an in-depth introduction of the Markov Decision Process, RL and its algorithms. Moreover, we present a literature review of the application of RL to a variety of fields, including robotics and autonomous control, communication and networking, natural language processing, games and self-organized system, scheduling management and configuration of resources, and computer vision.

show abstract

Design and Implementation of a Simulation System Based on Deep Q-Network for Mobile Actor Node Control in Wireless Sensor and Actor Networks

Cited by 42 publications

References 24 publications

Applications of Deep Reinforcement Learning in Communications and Networking: A Survey

Applications of Deep Reinforcement Learning in Communications and Networking: A Survey

Deep Reinforcement Learning for Autonomous Internet of Things: Model, Applications and Challenges

A Gentle Introduction to Reinforcement Learning and its Application in Different Fields

Contact Info

Product

Resources

About