Low-complexity scheduling policies for energy harvesting communication networks

Blasco, Pol; Gündüz, Deniz; Döhler, Mischa

doi:10.1109/isit.2013.6620497

Cited by 25 publications

(51 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The second class comprises online approaches [14] [15] [16]. Authors in [14] studied a multi-access wireless system with EH transmitters, and the access problem was modeled as a partially observable Markov decision process (POMDP).…”

Section: A Related Work and Motivationsmentioning

confidence: 99%

Reinforcement Learning-Based Multiaccess Control and Battery Prediction With Energy Harvesting in IoT Systems

Chu

Liao

et al. 2019

IEEE Internet Things J.

125

View full text Add to dashboard Cite

Energy harvesting (EH) is a promising technique to fulfill the long-term and self-sustainable operations for Internet of things (IoT) systems. In this paper, we study the joint access control and battery prediction problems in a small-cell IoT system including multiple EH user equipments (UEs) and one base station (BS) with limited uplink access channels. Each UE has a rechargeable battery with finite capacity. The system control is modeled as a Markov decision process without complete prior knowledge assumed at the BS, which also deals with large sizes in both state and action spaces. First, to handle the access control problem assuming causal battery and channel state information, we propose a scheduling algorithm that maximizes the uplink transmission sum rate based on reinforcement learning (RL) with deep Q-network (DQN) enhancement. Second, for the battery prediction problem, with a fixed round-robin access control policy adopted, we develop a RL based algorithm to minimize the prediction loss (error) without any model knowledge about the energy source and energy arrival process. Finally, the joint access control and battery prediction problem is investigated, where we propose a two-layer RL network to simultaneously deal with maximizing the sum rate and minimizing the prediction loss: the first layer is for battery prediction, the second layer generates the access policy based on the output from the first layer. Experiment results show that the three proposed RL algorithms can achieve better performances compared with existing benchmarks.

show abstract

Section: A Related Work and Motivationsmentioning

confidence: 99%

Reinforcement Learning-Based Multiaccess Control and Battery Prediction With Energy Harvesting in IoT Systems

Chu

Liao

et al. 2019

IEEE Internet Things J.

125

View full text Add to dashboard Cite

show abstract

“…The corresponding throughput maximization problems are formulated with partially observable Markov decision processes (POMDP) and cast into a restless multi-armed bandit. [14] and [15] show the optimality of a Round-Robin based myopic policy that schedules the K nodes with the largest beliefs to maximize the immediate reward under different system models. In [16], [17], under the infinite battery assumption, a uniformizing random ordered policy that selects the sensors based on a predefined random priority list and the outcome of transmissions in the previous time slot is shown to be asymptotically optimal in infinite horizon for a broad class of energy harvesting process.…”

Section: Related Workmentioning

confidence: 99%

“…Second, we assume that the statuses of batteries at sensor nodes are available at the fusion center, thus the optimization problem is actually a Markov decision process rather than a POMDP. Third, the optimality of the myopic policies in [14], [15], [23], [24] requires the energy harvesting processes to be uniform at sensors, while such assumption is not required for the optimality of our policy. Fourth, utilizing large deviation theory, we explicitly characterize the convergence rate of our policy.…”

Section: Related Workmentioning

confidence: 99%

Optimal Scheduling of Collaborative Sensing in Energy Harvesting Sensor Networks

Yang

2015

IEEE J. Select. Areas Commun.

View full text Add to dashboard Cite

In this paper, we consider a collaborative sensing scenario where sensing nodes are powered by energy harvested from ambient environment. In each time slot, an active sensor consumes one unit amount of energy to take an observation and transmit it back to a fusion center (FC). After receiving observations from all of the active sensors in a time slot, the FC aims to extract information from them. We assume that the sensing utility generated by the observations is a concave function of the number of the active sensing nodes in that slot. Our objective is to develop a sensing scheduling policy so that the time average utility generated by the sensors is maximized. We first consider an offline setting, where the energy harvesting profile over duration [0, T − 1] for each sensor is known beforehand. Assuming infinite battery capacity at sensors, we show that the optimal scheduling structure has a "majorization" property, and propose a procedure to construct a collaborative sensing policy with the identified structure explicitly. We then consider an online setting, under which the energy harvesting profile is available causally. Assuming the energy harvesting processes at individual sensors are independent but not necessarily identical Bernoulli processes, we show that the expected long-term time average sensing utility has an upper bound under any feasible scheduling policy satisfying the energy causality constraints. We then propose a randomized myopic policy, which aims to select a number of sensors with the highest energy levels to perform the sensing task in each slot. We show that the time average utility generated under the proposed policy converges to the upper bound almost surely as time T approaches infinity, thus it is optimal. The corresponding convergence rate is also explicitly characterized.

show abstract

“…In our work, the transmission parameters are obtained based on the users' statistical energy harvesting profiles and very few real-time message exchanges are required during data transmissions. We also notice that the authors of [22] studied online optimal scheduling for MAC, where the AP schedules a subset of the nodes over K orthogonal channels in each slot without the knowledge of the energy harvesting processes and users' battery states. They formulated the problem with a partially observable Markov decision process and showed that the myopic policy, equivalent to the round-robin policy, is optimal in two special cases.…”

Section: Introductionmentioning

confidence: 99%

“…Likewise, we study the optimal TDMA based transmission scheme for MAC based on the users' statistical energy harvesting information. In contrast to [22] [23], we assume that K users access the channel one by one in a roundrobin way. By convex optimization, we get a closed-form solution which indicates that equal-power TDMA is optimal for the infinite-capacity battery case and the optimal power is exactly equal to the average energy storage rate.…”

Section: Introductionmentioning

confidence: 99%

On Throughput Maximization of Time Division Multiple Access With Energy Harvesting Users

Liu

Dai

Chen

2016

IEEE Trans. Veh. Technol.

View full text Add to dashboard Cite

In this paper, we consider a multiple access channel, where multiple users equipped with energy harvesting batteries communicate to an access point. To avoid consuming extra energy on competition for the channel, the users are supposed to share the channel via Time Division Multiple Access (TDMA). In many existing works, it is commonly assumed that the users' energy harvesting processes and storage status are known to all the users before transmissions. In practice, such knowledge may not be readily available. To avoid excessive overhead for realtime information exchange, we consider the scenario where the users schedule their individual transmissions according to the users' statistical energy harvesting profiles. We first study the optimal transmission scheme in the case where each node has an infinite-capacity battery. By optimization theory, we show that to maximize the average system throughput, all the users should transmit at an identical optimal power, which solely depends on the energy harvesting rate per timeslot. We then study the equal-power TDMA scheme in the case where each node is equipped with a battery of finite capacity. The system is formulated as a polling system consisting of multiple energy queues and one server. By the Markov chain modeling method, we derive the performance of equal-power TDMA in this case, in terms of the energy loss ratio and average system throughput. In addition, we develop an algorithm to efficiently compute the optimal transmission power for each user in the finite-capacity battery case. We also consider an equal-time TDMA scheme, which assigns equal-length subslots to each user, and analyze its system performance. It is found that equal-power TDMA always outperforms equal-time TDMA in the infinite-capacity battery case, while equal-time TDMA exhibits compatible or even slightly better performance in some scenarios when the batteries have finite capacities.

show abstract

Low-complexity scheduling policies for energy harvesting communication networks

Cited by 25 publications

References 5 publications

Reinforcement Learning-Based Multiaccess Control and Battery Prediction With Energy Harvesting in IoT Systems

Reinforcement Learning-Based Multiaccess Control and Battery Prediction With Energy Harvesting in IoT Systems

Optimal Scheduling of Collaborative Sensing in Energy Harvesting Sensor Networks

On Throughput Maximization of Time Division Multiple Access With Energy Harvesting Users

Contact Info

Product

Resources

About