Optimal power allocation policy over two identical Gilbert-Elliott channels

Abstract

We study the fundamental problem of optimal power allocation over two identical Gilbert-Elliott (Binary Markov) communication channels. Our goal is to maximize the expected discounted number of bits transmitted over an infinite time span by judiciously choosing one of the four actions for each time slot: 1) allocating power equally to both channels, 2) allocating all the power to channel 1, 3) allocating all the power to channel 2, and 4) allocating no power to any of the channels. As the channel state is unknown when power allocation decision is made, we model this problem as a partially observable Markov decision process(POMDP), and derive the optimal policy which gives the optimal action to take under different possible channel states. Two different structures of the optimal policy are derived analytically and verified by linear programming simulation. We also illustrate how to construct the optimal policy by the combination of threshold calculation and linear programming simulation once system parameters are known.

Optimal Power Allocation over Multiple Identical Gilbert-Elliott Channels

“…In [9], the authors study the problem of choosing a transmitting strategy from two choices emphasizing the case when the channel transition probabilities are unknown. The work in [10] and [11] is most relevant to the work in this paper, the differences between these three are as follows: [10] addresses power allocation problem in the context of two identical channels and three allocation strategies: betting on channel 1, betting on channel 2 and using both channels, whilst [11] added one more action of using none of the channels and introduced penalty caused by transmission on a bad channel. The spirit of this paper is similar to those in [10] and [11], but addresses a more challenging setting involving N identical channels(N ≥ 3).…”

“…The work in [10] and [11] is most relevant to the work in this paper, the differences between these three are as follows: [10] addresses power allocation problem in the context of two identical channels and three allocation strategies: betting on channel 1, betting on channel 2 and using both channels, whilst [11] added one more action of using none of the channels and introduced penalty caused by transmission on a bad channel. The spirit of this paper is similar to those in [10] and [11], but addresses a more challenging setting involving N identical channels(N ≥ 3). When N is large, the power allocation decisions becomes much more complicated, and it is more difficult to derive and express the optimal policy.…”

Partially Observable Markov Decision Process-Based Transmission Policy over Ka-Band Channels for Space Information Networks

“…by solving the value function Equation (18). Else, if λ 0 < ρ 1 < λ 1 < ρ 2 , as is shown in Figure 4e, then we have a i = C for s i−1 = 0 where (18), and a i = O is the optimal action for s i−1 = 1 where…”

“…In previous studies [17][18][19], the time-varying rain attenuation at the Ka-band channel is used to model to a two-state Gilbert-Elliot (GE) channel, and several works have focused on the optimal data transmission policy. In [20], three data transmission actions were proposed to be chosen at the beginning of each time slot to maximize the expected long-term throughput.…”