Autonomous Management of Energy-Harvesting IoT Nodes Using Deep Reinforcement Learning

Murad, Abdulmajid; Kraemer, Frank Alexander; Bach, Kerstin; Taylor, Gavin

doi:10.1109/saso.2019.00015

Cited by 21 publications

(18 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It employs a reward function and learns through interaction of an agent with its environment, with no need for a complete control model or explicit supervision [22]. An RL agent is trained to improve a task by learning from experience, that is, interacting with that particular task in context [62]. The algorithm is trained with the goal to maximize the cumulative reward.…”

Section: ) Machine Learning (Ml)mentioning

confidence: 99%

See 1 more Smart Citation

Autonomous IoT Device Management Systems: Structured Review and Generalized Cognitive Model

Bråten

Kraemer

Palma

2021

IEEE Internet Things J.

Self Cite

View full text Add to dashboard Cite

Research on autonomous management for largescale deployments of constrained devices is still a maturing field in the Internet of Things (IoT). Although much research has been conducted on how to achieve autonomous management in specific cases, there is a need for literature investigating which mechanisms can achieve such behavior in a generalized way. In this review, we present a comprehensive and structured study of the mechanisms for autonomous device management of constrained IoT devices in the light of management tasks, operational environment, network topology, resource constraints, scalability and management categories. Data extracted from 32 relevant cases is first organized and analyzed according to a synthesized taxonomy of observed adaptation mechanisms, and then combined with state-of-the-art models of autonomous operations, identifying common patterns for autonomous management. Based on our findings we substantiate best practices for designing and implementing solutions around adaptation mechanisms. We then present a generalized model for autonomous device management that describes and explains the processes required for autonomous operation, unifying the insights from previous works as one cohesive archetype.

show abstract

Section: ) Machine Learning (Ml)mentioning

confidence: 99%

“…An example can be seen in [37], where Edalat et al use reinforcement learning for network lifetime optimization. Challenges with RL are the design of the reward function, as this requires an in-depth knowledge of the domain and the system goals, as well as a potentially high training effort [62].…”

Section: ) Machine Learning (Ml)mentioning

confidence: 99%

Autonomous IoT Device Management Systems: Structured Review and Generalized Cognitive Model

Bråten

Kraemer

Palma

2021

IEEE Internet Things J.

Self Cite

View full text Add to dashboard Cite

show abstract

“…For example, in an energy-harvesting management system, PPO algorithm [292] is used to control IoT nodes for power allocation. The action space, as stated in [245], is sampled from a Gaussian distribution to denote the load of each node ranging from 0% to 100%. Similarly, in another work [9] that studied energy harvesting WSNs, the Actor-Critic [179] algorithm is implemented to control the packet rate during transmission.…”

Section: Rl Categorizationmentioning

confidence: 99%

Orchestrating the Development Lifecycle of Machine Learning-based IoT Applications

Qian

Wen

et al. 2020

ACM Comput. Surv.

View full text Add to dashboard Cite

Machine Learning (ML) and Internet of Things (IoT) are complementary advances: ML techniques unlock the potential of IoT with intelligence, and IoT applications increasingly feed data collected by sensors into ML models, thereby employing results to improve their business processes and services. Hence, orchestrating ML pipelines that encompass model training and implication involved in the holistic development lifecycle of an IoT application often leads to complex system integration. This paper provides a comprehensive and systematic survey of the development lifecycle of ML-based IoT applications. We outline the core roadmap and taxonomy, and subsequently assess and compare existing standard techniques used at individual stages.

show abstract

“…It can successfully control more than 20 kinds of physics tasks such as cart-pole swing-up, legged locomotion, car driving, and Reacher domain with multiple continuous action spaces (Lillicrap et al, 2016). In engineering, DRL has been widely used in optimization and control problems in practical applications such as robotics (Gu et al, 2017), HAVC control (Chen et al, 2018), and energy harvesting (Long & Büyüköztürk, 2020;Murad et al, 2019). These successes demonstrate the ability of DRL to learn complex tasks that require expert-level knowledge and experience.…”

Section: Introductionmentioning

confidence: 99%

Deep reinforcement learning for automated design of reinforced concrete structures

Jeong

2021

Computer aided Civil Eng

View full text Add to dashboard Cite

This study proposes a novel concept of reinforcement learning (RL) framework to facilitate automated structural design, with a particular focus on reinforced concrete (RC) beam design as case studies. We trained an RL agent called deep deterministic policy gradient (DDPG) with a convolutional neural network function approximators. The RL agent was successfully trained to design RC beams subject to American Concrete Institute (318) provisions without any hand-labeled dataset. A python-based RC beam design environment was developed and used to simulate RC beam designs with customized reward functions that encouraged the agent to minimize material cost by maximizing reward. The DDPG agent self-learned cost-effective RC beam design through 100,000 randomly generated design cases during the training procedure. The trained agent was able to design an RC beam in a cost-effective way while taking both flexural and shear reinforcement arrangements into consideration. The trained agent generated nearoptimal design parameters without the need for unnecessary iterations over various design conditions. The performance of the agent was validated with 100 design cases and comparative studies, showing a great promise for automated RC beam design.

show abstract

Autonomous Management of Energy-Harvesting IoT Nodes Using Deep Reinforcement Learning

Cited by 21 publications

References 17 publications

Autonomous IoT Device Management Systems: Structured Review and Generalized Cognitive Model

Autonomous IoT Device Management Systems: Structured Review and Generalized Cognitive Model

Orchestrating the Development Lifecycle of Machine Learning-based IoT Applications

Deep reinforcement learning for automated design of reinforced concrete structures

Contact Info

Product

Resources

About