An Online Learning Approach to Network Application Optimization with Guarantee

Cai, Kechao; Liu, Xutong; Chen, Yu-Zhen Janice; Lui, John C. S.

doi:10.1109/infocom.2018.8485906

Cited by 24 publications

(5 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Third, they assume that all the arms are available at all times. Last but not least, the proof techniques for regret analysis in [24], [25] are very different from ours.…”

Section: Related Workmentioning

confidence: 82%

“…Remark: Note that the work of [24] also studies an MAB problem with minimum-guarantee constraints. However, their work differs significantly from ours because their considered minimum guarantee is for the total rewards (of some type/level) rather than for each individual arm, i.e., fairness among arms is not modeled.…”

Section: A Feasibility Optimalitymentioning

confidence: 99%

“…However, their work differs significantly from ours because their considered minimum guarantee is for the total rewards (of some type/level) rather than for each individual arm, i.e., fairness among arms is not modeled. More importantly, the proposed learning algorithm in [24] may violate the constraints. Although they show that the violations are upper bounded by O(T 5/6 ), this upper bound implies that the constraints may not be satisfied even after a long enough time.…”

Section: A Feasibility Optimalitymentioning

confidence: 99%

“…Hence, these types of constraints are very different from the long-term fairness constraints we consider in this paper. Some very recent work considers multi-type rewards [23] and multi-level rewards [24], [25]. They introduce a minimum guarantee requirement that the total reward of some type/level must be no smaller than a given threshold.…”

Section: Related Workmentioning

confidence: 99%

“…The required minimum guarantee is for the total rewards (of some type/level) rather than for each individual arm. Second, no learning algorithm is proposed in [23]; the proposed learning algorithms in [24], [25] may violate the constraints, although they show provable violation bounds. Third, they assume that all the arms are available at all times.…”

Section: Related Workmentioning

confidence: 99%

See 4 more Smart Citations

Combinatorial Sleeping Bandits With Fairness Constraints

Liu

2020

IEEE Trans. Netw. Sci. Eng.

View full text Add to dashboard Cite

The multi-armed bandit (MAB) model has been widely adopted for studying many practical optimization problems (network resource allocation, ad placement, crowdsourcing, etc.) with unknown parameters. The goal of the player (i.e., the decision maker) here is to maximize the cumulative reward in the face of uncertainty. However, the basic MAB model neglects several important factors of the system in many realworld applications, where multiple arms (i.e., actions) can be simultaneously played and an arm could sometimes be "sleeping" (i.e., unavailable). Besides reward maximization, ensuring fairness is also a key design concern in practice. To that end, we propose a new Combinatorial Sleeping MAB model with Fairness constraints, called CSMAB-F, aiming to address the aforementioned crucial modeling issues. The objective is now to maximize the reward while satisfying the fairness requirement of a minimum selection fraction for each individual arm. To tackle this new problem, we extend an online learning algorithm, called Upper Confidence Bound (UCB), to deal with a critical tradeoff between exploitation and exploration and employ the virtual queue technique to properly handle the fairness constraints. By carefully integrating these two techniques, we develop a new algorithm, called Learning with Fairness Guarantee (LFG), for the CSMAB-F problem. Further, we rigorously prove that not only LFG is feasibility-optimal, but it also has a time-average regret upper bounded by N 2η, where N is the total number of arms, m is the maximum number of arms that can be simultaneously played, T is the time horizon, β1 and β2 are constants, and η is a design parameter that we can tune. Finally, we perform extensive simulations to corroborate the effectiveness of the proposed algorithm. Interestingly, the simulation results reveal an important tradeoff between the regret and the speed of convergence to a point satisfying the fairness constraints.

show abstract

“…Third, they assume that all the arms are available at all times. Last but not least, the proof techniques for regret analysis in [24], [25] are very different from ours.…”

Section: Related Workmentioning

confidence: 82%

Section: A Feasibility Optimalitymentioning

confidence: 99%

Section: A Feasibility Optimalitymentioning

confidence: 99%