On a restless multi-armed bandit problem with non-identical arms

Nayyar, Naumaan; Gai, Yi; Krishnamachari, Bhaskar

doi:10.1109/allerton.2011.6120191

Cited by 6 publications

(7 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Knowing that this problem has a 0 or 2 threshold structure reduces the problem of identifying optimal performance to finding the (only up to 2) threshold parameters. In settings where the underlying state transition matrices are unknown, this could be exploited by using a multiarmed bandit (MAB) formulation to find the best possible thresholds (similar to the ideas in the papers [9] and [10]). Also, we would like to investigate the case of non-identical channels, and derive useful results for more than 2 channels, possibly in the form of computing the Whittle index [17], if computing the optimal policy in general turns out to be intractable.…”

Section: Discussionmentioning

confidence: 99%

“…While similar in spirit to these two studies, our work addresses a more challenging setting involving two independent channels. A more related twochannel problem is studied in [10], which characterizes the optimal policy to opportunistically access two non-identical Gilber-Elliott channels (generalizing the prior work on sensing policies for identical channels [6], [7]). While we address only identical channels in this work, the strategy space explored here is richer because in our formulation of power allocation, it is possible to use both channels simultaneously whilst in [6], [7], [10] only one channel is accessed in each time slot.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Power allocation over two identical Gilbert-Elliott channels

Tang

Mansourifard

Krishnamachari

2013

2013 IEEE International Conference on Communications (ICC)

Self Cite

View full text Add to dashboard Cite

We study the problem of power allocation over two identical Gilbert-Elliot communication channels. Our goal is to maximize the expected discounted number of bits transmitted over an infinite time horizon. This is achieved by choosing among three possible strategies: (1) betting on channel 1 by allocating all the power to this channel, which results in high data rate if channel 1 happens to be in good state, and zero bits transmitted if channel 1 is in bad state (even if channel 2 is in good state) (2) betting on channel 2 by allocating all the power to the second channel, and (3) a balanced strategy whereby each channel is allocated half the total power, with the effect that each channel can transmit a low data rate if it is in good state. We assume that each channel's state is only revealed upon transmission of data on that channel. We model this problem as a partially observable Markov decision processes (MDP), and derive key threshold properties of the optimal policy. Further, we show that by formulating and solving a relevant linear program the thresholds can be determined numerically when system parameters are known.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Power allocation over two identical Gilbert-Elliott channels

Tang

Mansourifard

Krishnamachari

2013

2013 IEEE International Conference on Communications (ICC)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Further, we would like to find a closed form expression for the boundary of action region. Also, we would like to investigate the case of non-identical channels like [16], or derive useful results for more than 2 channels.…”

Section: Discussionmentioning

confidence: 99%

Optimal power allocation policy over two identical Gilbert-Elliott channels

Jiang

Tang

Krishnamachari

2013

2013 IEEE International Conference on Communications (ICC)

Self Cite

View full text Add to dashboard Cite

We study the fundamental problem of optimal power allocation over two identical Gilbert-Elliott (Binary Markov) communication channels. Our goal is to maximize the expected discounted number of bits transmitted over an infinite time span by judiciously choosing one of the four actions for each time slot: 1) allocating power equally to both channels, 2) allocating all the power to channel 1, 3) allocating all the power to channel 2, and 4) allocating no power to any of the channels. As the channel state is unknown when power allocation decision is made, we model this problem as a partially observable Markov decision process(POMDP), and derive the optimal policy which gives the optimal action to take under different possible channel states. Two different structures of the optimal policy are derived analytically and verified by linear programming simulation. We also illustrate how to construct the optimal policy by the combination of threshold calculation and linear programming simulation once system parameters are known.

show abstract

“…We specifically simulate links following the Gilbert-Elliot (GE) model [55], [56]. The GE model, consisting of two states (see Fig.…”

Section: Appendix C Correlated Link Lossesmentioning

confidence: 99%

<italic>RT-ByzCast</italic>: Byzantine-Resilient Real-Time Reliable Broadcast

Kozhaya

Decouchant

Esteves-Veríssimo

2019

IEEE Trans. Comput.

View full text Add to dashboard Cite

Today's cyber-physical systems face various impediments to achieving their intended goals, namely, communication uncertainties and faults, relative to the increased integration of networked and wireless devices, hinder the synchronism needed to meet real-time deadlines. Moreover, being critical, these systems are also exposed to significant security threats. This threat combination increases the risk of physical damage. This paper addresses these problems by studying how to build the first real-time Byzantine reliable broadcast protocol (RTBRB) tolerating network uncertainties, faults, and attacks. Previous literature describes either real-time reliable broadcast protocols, or asynchronous (non real-time) Byzantine ones.We first prove that it is impossible to implement RTBRB using traditional distributed computing paradigms, e.g., where the error/failure detection mechanisms of processes are decoupled from the broadcast algorithm itself, even with the help of the most powerful failure detectors. We circumvent this impossibility by proposing RT-ByzCast, an algorithm based on aggregating digital signatures in a sliding time-window and on empowering processes with self-crashing capabilities to mask and bound losses. We show that RT-ByzCast (i) operates in real-time by proving that messages broadcast by correct processes are delivered within a known bounded delay, and (ii) is reliable by demonstrating that correct processes using our algorithm crash themselves with a negligible probability, even with message loss rates as high as 60%.

show abstract

On a restless multi-armed bandit problem with non-identical arms

Cited by 6 publications

References 16 publications

Power allocation over two identical Gilbert-Elliott channels

Power allocation over two identical Gilbert-Elliott channels

Optimal power allocation policy over two identical Gilbert-Elliott channels

<italic>RT-ByzCast</italic>: Byzantine-Resilient Real-Time Reliable Broadcast

Contact Info

Product

Resources

About