“…This has been a very successful heuristic for restless bandits, which, while suboptimal in general, is provably optimal in an asymptotic sense [26], [27] and has good empirical performance. It and its variants have been used extensively in logistical and engineering applications, some recent instances of the latter in communications and control being for sensor scheduling [19], multi-UAV coordination [20], congestion control [3], [4], [13], channel allocation in wireless networks [14], cognitive radio [17] and real-time wireless multicast [23]. Book length treatments of indexable restless bandits appear in [12], [24].…”