Risk-Averse Allocation Indices for Multiarmed Bandit Problem

Malekipirbazari, Milad; Çavuş, Özlem

doi:10.1109/tac.2021.3053539

Cited by 8 publications

(3 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Related Work: As a well-known model to capture the exploration-exploitation dilemma in decision-making, multiarmed bandits have attracted extensive attention (see [13] for a survey). A variety of generalized bandit problems have been investigated, where the situations with non-stationary reward functions [14], restless arms [15], satisficing reward objectives [16], heavy-tailed reward distributions [17], risk-averse decision-makers [18], and multiple players [19] are considered. Distributed algorithms have also been proposed to tackle bandit problems (e.g., see [20]- [27]).…”

Section: Introductionmentioning

confidence: 99%

Non-Stationary Representation Learning in Sequential Linear Bandits

Qin

Menara

Oymak

et al. 2022

IEEE Open J. Control. Syst.

View full text Add to dashboard Cite

In this paper, we study representation learning for multi-task decision-making in non-stationary environments. We consider the framework of sequential linear bandits, where the agent performs a series of tasks drawn from different environments. The embeddings of tasks in each environment share a lowdimensional feature extractor called representation, and representations are different across environments. We propose an online algorithm that facilitates efficient decision-making by learning and transferring nonstationary representations in an adaptive fashion. We prove that our algorithm significantly outperforms the existing ones that treat tasks independently. We also conduct experiments using both synthetic and real data to validate our theoretical insights and demonstrate the efficacy of our algorithm.

show abstract

Section: Introductionmentioning

confidence: 99%

Non-Stationary Representation Learning in Sequential Linear Bandits

Qin

Menara

Oymak

et al. 2022

IEEE Open J. Control. Syst.

View full text Add to dashboard Cite

show abstract

“…As a well-known model to capture the exploration-exploitation dilemma in decisionmaking, multi-armed bandits have attracted extensive attention (see [13] for a survey). A variety of generalized bandit problems have been investigated, where the situations with non-stationary reward functions [14], restless arms [15], satisficing reward objectives [16], heavy-tailed reward distributions [17], risk-averse decision-makers [18], and multiple players [19] are considered. Recently, distributed algorithms have also been proposed to tackle bandit problems (e.g., see [20][21][22][23][24][25][26][27]).…”

Section: Introductionmentioning

confidence: 99%

Non-Stationary Representation Learning in Sequential Linear Bandits

Qin¹,

Menara²,

Oymak³

et al. 2022

Preprint

View full text Add to dashboard Cite

In this paper, we study representation learning for multi-task decision-making in non-stationary environments. We consider the framework of sequential linear bandits, where the agent performs a series of tasks drawn from distinct sets associated with different environments. The embeddings of tasks in each set share a low-dimensional feature extractor called representation, and representations are different across sets. We propose an online algorithm that facilitates efficient decision-making by learning and transferring non-stationary representations in an adaptive fashion. We prove that our algorithm significantly outperforms the existing ones that treat tasks independently. We also conduct experiments using both synthetic and real data to validate our theoretical insights and demonstrate the efficacy of our algorithm.

show abstract

“…The Upper Confidence Bound (UCB) algorithm and its variants have proven their strength in tackling multi-armed bandit problems (e.g., see [5], [6]). Various generalizations of the classical bandit problem have been studied, in which nonstationary reward functions [7], [8], restless arms [9], satisficing reward objectives [10], risk-averse decision-makers [11], heavy-tailed reward distributions [12], and multiple players [13] are considered. Recently, increasing attention has been also paid to tackling bandit problems in a distributed fashion (e.g., see [14]- [17]).…”

Section: Introductionmentioning

confidence: 99%

Representation Learning for Context-Dependent Decision-Making

Qin¹,

Menara²,

Oymak³

et al. 2022

Preprint

View full text Add to dashboard Cite

Humans are capable of adjusting to changing environments flexibly and quickly. Empirical evidence has revealed that representation learning plays a crucial role in endowing humans with such a capability. Inspired by this observation, we study representation learning in the sequential decision-making scenario with contextual changes. We propose an online algorithm that is able to learn and transfer contextdependent representations and show that it significantly outperforms the existing ones that do not learn representations adaptively. As a case study, we apply our algorithm to the Wisconsin Card Sorting Task, a well-established test for the mental flexibility of humans in sequential decision-making. By comparing our algorithm with the standard Q-learning and Deep-Q learning algorithms, we demonstrate the benefits of adaptive representation learning. Y. Qin and F. Pasqualetti are with the Department of Mechanical Engineering (

show abstract

Risk-Averse Allocation Indices for Multiarmed Bandit Problem

Cited by 8 publications

References 23 publications

Non-Stationary Representation Learning in Sequential Linear Bandits

Non-Stationary Representation Learning in Sequential Linear Bandits

Non-Stationary Representation Learning in Sequential Linear Bandits

Representation Learning for Context-Dependent Decision-Making

Contact Info

Product

Resources

About