Near-optimal reinforcement learning framework for energy-aware sensor communications

Pandana, C.; Liu, K.J.R.

doi:10.1109/jsac.2005.843547

Cited by 66 publications

(37 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Additional routing protocols based on reinforcement learning, together with their properties are discussed in (Di & Joo, 2007;Kulkarni et al, 2009;Predd et al, 2006). Examples of applying reinforcement learning to medium access are available in (Liu & Elahanany, 2006;Pandana & Liu, 2005). Another candidate for improving routing performance in WSNs is swarm intelligence.…”

Section: Conclusion and Further Readingmentioning

confidence: 99%

Machine Learning across the WSN Layers

Förster¹,

Murphy²

2011

Emerging Communications for Wireless Sensor Networks

View full text Add to dashboard Cite

Wireless sensor networks (WSNs) have seen rapid research and industrial development in recent years. Both the costs and size of individual nodes have been constantly decreasing, opening new opportunities for a wide range of applications. Nevertheless, designing software to achieve energy-efficient, robust and flexible data dissemination remains an open problem with many competing solutions. In parallel, researchers have effectively exploited machine learning techniques to achieve efficient solutions in environments with distribution and rapidly fluctuating properties, analogous to WSN domains. Applying machine learning techniques to WSNs inherently has the potential to improve the robustness and flexibility of communications and data processing, while simultaneously optimizing energy expenditure. This chapter concentrates on applications of machine learning at all layers in the WSN network stack. First, it provides a brief background and summary of three of the most commonly used machine learning techniques: reinforcement learning, neural networks and decision trees. Then, it uses example research from the literature to describe current efforts at each level of the stack, and outlines future opportunities.

show abstract

Section: Conclusion and Further Readingmentioning

confidence: 99%

Machine Learning across the WSN Layers

Förster¹,

Murphy²

2011

Emerging Communications for Wireless Sensor Networks

View full text Add to dashboard Cite

show abstract

“…The method is based on the distributed reinforcement learning routing algorithm [17]. We note that the reinforcement learning algorithm has been shown to be an effective online decision making procedure in sensor network application [18]. The resulting algorithm can be characterized as a version of distributed Bellman-Ford algorithm that performs its path relaxation step asynchronously and online with the edge cost defined as weighted energy required to transmit packet in that hop [16].…”

Section: Distributed Implementation and Learning Algorithmmentioning

confidence: 99%

Robust connectivity-aware energy-efficient routing for wireless sensor networks

Pandana¹,

Liu

2008

IEEE Trans. Wireless Commun.

Self Cite

View full text Add to dashboard Cite

Abstract-In this paper, we consider a class of energy-aware routing algorithm that explicitly takes into account the connectivity of the remaining sensor network. In typical sensor network deployments, some nodes may be more important than other nodes because the failure of these nodes causes the network disintegration, which results in early termination of information delivery. To mitigate this problem, we propose a class of routing algorithms called keep-connect algorithms, that use computable measures of network connectivity in determining how to route packets. The proposed algorithms embed the importance of the nodes in the routing cost/metric. The importance of a node is characterized by the algebraic connectivity of the remaining graph when that node fails. We prove several properties of the proposed routing algorithm including the energy consumption upper bound. Using extensive simulations, we demonstrate that the proposed algorithm achieves significant performance improvement compared to the existing routing algorithms. More importantly, we show that our proposed algorithm is more robust in terms of algebraic network connectivity compared to the existing algorithm. Finally, we present the distributed implementation of our proposed algorithm.

show abstract

“…Therefore, it is more adaptive and realistic to solve the problems that can be formulated as Markov decision process (MDP). Hence RL approach has been widely used to solve the problems of cross-layer design for wireless communication, such as call admission control [8][9][10], energy-aware sensor communications [11], cell configuration [12] and congestion control [13].…”

Section: Introductionmentioning

confidence: 99%

Adaptive transmission scheduling over fading channels for energy-efficient cognitive radio networks by reinforcement learning

et al. 2009

View full text Add to dashboard Cite

In this paper, we address a cross-layer issue of long-term average utility maximization in energy-efficient cognitive radio networks supporting packetized data traffic under the constraint of collision rate with licensed users. Utility is determined by the number of packets transmitted successfully per consumed power and buffer occupancy. We formulate the problem by dynamic programming method namely constrained Markov decision process (CMDP). Reinforcement learning (RL) approach is employed to finding a near-optimal policy under undiscovered environment. The policy learned by RL can guide transmitter to access available channels and select proper transmission rate at the beginning of each frame for its long-term optimal goals. Some implement problems of the RL approach are discussed. Firstly, state space compaction is utilized to cope with so-called curse of dimensionality due to large state space of formulated CMDP. Secondly, action set reduction is presented to reduce the number of actions for some system states. Finally, the CMDP is converted to a corresponding unconstrained Markov decision process (UMDP) by Lagrangian multiplier approach and a golden section search method is proposed to find the proper multiplier. In order to evaluate the performance of the policy learned by RL, we present two naive policies and compare them by simulations.

show abstract

Near-optimal reinforcement learning framework for energy-aware sensor communications

Cited by 66 publications

References 13 publications

Machine Learning across the WSN Layers

Machine Learning across the WSN Layers

Robust connectivity-aware energy-efficient routing for wireless sensor networks

Adaptive transmission scheduling over fading channels for energy-efficient cognitive radio networks by reinforcement learning

Contact Info

Product

Resources

About