Adaptive Duty Cycling in Sensor Networks With Energy Harvesting Using Continuous-Time Markov Chain and Fluid Models

Chan, Wai Hong Ronald; Zhang, Peng-Fei; Nevat, Ido; Nagarajan, Sai Ganesh; Valera, Alvin; Tan, Hwee-Xian; Gautam, Natarajan

doi:10.1109/jsac.2015.2478717

Cited by 34 publications

(13 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…More recently, reinforcement learning (RL) algorithms have been proposed as a more flexible and powerful approach to achieving node level energy neutrality in energy harvesting sensor networks. In Chan et al (2015) the challenge of maximising quality of service while maintaining battery reserves is framed as a continuous time markov decision process, and solved using tabular Q-learning. Hsu, Liu, & Wang (2014) propose a tabular Q-learning approach for query driven wireless sensor networks.…”

Section: Related Workmentioning

confidence: 99%

“…If 0 % duty cycle periods occur simultaneously at all nodes in the network, any events occurring in this period will go undetected. Tabular reinforcement learning algorithms as proposed in (Chan et al, 2015), (Hsu et al, 2014), (Shresthamali et al, 2017) are likely to be intractable when considering the entire network, due to the size of the state-action space, and so more powerful reinforcement learning approaches are required, which utilise function approximators instead of lookup tables, as in (Mnih et al, 2015), (Mnih et al, 2016), (Peters & Schaal, 2008). Deep neural network based approaches have also been proposed for civil and structural monitoring problems recently, outside the context of reinforcement learning, as in Y.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Collaborative duty cycling strategies in energy harvesting sensor networks

Long

Büyüköztürk

2019

Computer aided Civil Eng

View full text Add to dashboard Cite

Energy harvesting wireless sensor networks are a promising solution for low cost, long lasting civil monitoring applications. But management of energy consumption is a critical concern to ensure these systems provide maximal utility. Many common civil applications of these networks are fundamentally concerned with detecting and analysing infrequently occurring events. To conserve energy in these situations, a subset of nodes in the network can assume active duty, listening for events of interest, while the remaining nodes enter low power sleep mode to conserve battery. However, judicious planning of the sequence of active node assignments is needed to ensure that as many nodes as possible can be reached upon the detection of an event, and that the system maintains capability in times of low energy harvesting capabilities. In this paper, we propose a novel reinforcement learning agent which acts as a centralised power manager for this system. We develop a comprehensive simulation environment to emulate the behaviour of an energy harvesting sensor network, with consideration of spatially varying energy harvesting capabilities, and wireless connectivity. We then train the proposed reinforcement learning agent to learn optimal node selection strategies through interaction with the simulation environment. The behaviour and performance of these strategies are tested on real unseen solar energy data, to demonstrate the efficacy of the method. The deep reinforcement learning agent is shown to outperform baseline approaches on both seen and unseen data.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Collaborative duty cycling strategies in energy harvesting sensor networks

Long

Büyüköztürk

2019

Computer aided Civil Eng

View full text Add to dashboard Cite

show abstract

“…Adaptive duty cycling schemes focus more on achieving energy efficiency while fulfilling some QoS parameters such as throughput and delay [25], [26]. Achieving a low duty cycle results in high energy saving and in most cases, it leads to increased delay [4].…”

Section: Introductionmentioning

confidence: 99%

Queue and Priority-Aware Adaptive Duty Cycle Scheme for Energy Efficient Wireless Sensor Networks

et al. 2020

View full text Add to dashboard Cite

In Wireless Sensor Networks (WSN), energy efficiency is a fundamental issue that requires attention in the design of communication protocols. Energy waste occurs as a result of collision and idle listening. The widely used mechanism for energy saving in WSN is duty cycling. Duty cycling coordinates the sleep/wake-up time of sensor nodes to maximize the network lifetime while achieving specific application goals such as high throughput or low latency. Most existing works focused more on static duty cycle, which cannot guarantee the desired end-to-end delay at varying network conditions. In applications with specified delay requirements, the duty cycle of every node should be adjusted separately at runtime depending on the network conditions to achieve the desired delay and energy efficiency. In this paper, we present an Energy Efficient and QoS-aware (EEQ) MAC protocol with a duty cycle scheme that adapts the node's duty cycle to the queue size and priority class of a packet to reduce the delay of high priority packets and support timebounded delivery of priority packets. By checking the queue size and the priority class of packets in the message queue of each node, the node determines whether or not to adjust its duty cycle. In this approach, a node increases the length of its active period in the event of high traffic which provides less waiting time for the packets in the queue. The sender node informs the receiver the duration for which it has to stay awake at the beginning of data transmission, both the sending and the receiver's duty cycle is controlled based on the queue length and the priority of the packets. This approach saves energy and lessen packet latency. Finally, extensive simulation experiments were conducted to evaluate its energy performance within different network topologies. Comparisons with the existing energy-aware MAC protocol verified the effect of EEQ on improving the energy efficiency and extending the lifespan of WSNs.

show abstract

“…Therefore, researchers have made great effort to reduce energy consumption and to prolong network lifetime in various application fields [1][2][3][4][5][6][7]. Recently, energy harvesting technologies have been proposed [8] and various problems in energy harvesting sensor networks have been studied [9][10][11][12][13][14][15][16][17][18][19][20][21][22].…”

Section: Introductionmentioning

confidence: 99%

“…The bitreversal permutation sequence can also be used to achieve this aim [17]. In [18], a new framework was developed to 2 International Journal of Distributed Sensor Networks model the adaptive duty cycling in energy harvesting sensor networks as a Markov Decision Process. To achieve optimal performance, an optimal scheduling algorithm was proposed in [19] which considered the residual energy to achieve closeto-optimal utility.…”

Section: Introductionmentioning

confidence: 99%

Efficient Sleep Scheduling Algorithm for Target Tracking in Double-Storage Energy Harvesting Sensor Networks

Chen

Zeng

Zhao

2016

International Journal of Distributed Sensor Networks

View full text Add to dashboard Cite

Target tracking is a typical application in wireless sensor networks. Both energy efficiency and tracking performance are important issues that need to be considered. They are a pair of contradictions most of the time. Saving energy often sacrifices tracking performance, while enhancing tracking performance needs to consume more energy. In this paper, an efficient sleep scheduling algorithm is put forward to tackle the above problem in energy harvesting sensor networks. At first, we modify the probabilitybased prediction and sleep scheduling (PPSS) algorithm to track the target and further use another sleep scheduling algorithm we proposed to wake tracking nodes when the target is likely to be missed (i.e., it is unsuccessful to wake next-moment tracking nodes). Secondly, a double-storage energy harvesting architecture is employed to increase residual energy of sensor nodes and to extend network lifetime. Simulation results reveal that the proposed sleep scheduling algorithm can improve tracking performance and prolong network lifetime compared with the PPSS algorithm and the proposed algorithm without energy harvesting.

show abstract

Adaptive Duty Cycling in Sensor Networks With Energy Harvesting Using Continuous-Time Markov Chain and Fluid Models

Cited by 34 publications

References 37 publications

Collaborative duty cycling strategies in energy harvesting sensor networks

Collaborative duty cycling strategies in energy harvesting sensor networks

Queue and Priority-Aware Adaptive Duty Cycle Scheme for Energy Efficient Wireless Sensor Networks

Efficient Sleep Scheduling Algorithm for Target Tracking in Double-Storage Energy Harvesting Sensor Networks

Contact Info

Product

Resources

About