Mechanisms with learning for stochastic multi-armed bandit problems

Jain, Shweta; Bhat, Satyanath; Ghalme, Ganesh; Padmanabhan, Divya; Narahari, Y.

doi:10.1007/s13226-016-0186-3

Cited by 6 publications

(4 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thompson sampling was arbitrarily selected as the policy tool for this case study using the Multi-Armed Bandit (MAB) approach, owing to its adept balance between exploration and exploitation, as well as its parameter-free nature. Thompson sampling is particularly useful in scenarios where action rewards follow a Bernoulli distribution, distinguishing successes from failures [12]. This policy is specifically tailored to tackle online decision problems, including the Multi-Armed Bandit problem.…”

Section: The Mab-problem-based Algorithmmentioning

confidence: 99%

Exploring Multi-Armed Bandit (MAB) as an AI Tool for Optimising GMA-WAAM Path Planning

Ferreira,

Schubert,

Scotti

2024

JMMP

View full text Add to dashboard Cite

Conventional path-planning strategies for GMA-WAAM may encounter challenges related to geometrical features when printing complex-shaped builds. One alternative to mitigate geometry-related flaws is to use algorithms that optimise trajectory choices—for instance, using heuristics to find the most efficient trajectory. The algorithm can assess several trajectory strategies, such as contour, zigzag, raster, and even space-filling, to search for the best strategy according to the case. However, handling complex geometries by this means poses computational efficiency concerns. This research aimed to explore the potential of machine learning techniques as a solution to increase the computational efficiency of such algorithms. First, reinforcement learning (RL) concepts are introduced and compared with supervised machining learning concepts. The Multi-Armed Bandit (MAB) problem is explained and justified as a choice within the RL techniques. As a case study, a space-filling strategy was chosen to have this machining learning optimisation artifice in its algorithm for GMA-AM printing. Computational and experimental validations were conducted, demonstrating that adding MAB in the algorithm helped to achieve shorter trajectories, using fewer iterations than the original algorithm, potentially reducing printing time. These findings position the RL techniques, particularly MAB, as a promising machining learning solution to address setbacks in the space-filling strategy applied.

show abstract

Section: The Mab-problem-based Algorithmmentioning

confidence: 99%

Exploring Multi-Armed Bandit (MAB) as an AI Tool for Optimising GMA-WAAM Path Planning

Ferreira,

Schubert,

Scotti

2024

JMMP

View full text Add to dashboard Cite

show abstract

“…When online learning is involved, the stochastic multiarmed bandit (MAB) problem captures the exploration vs. exploitation trade-off effectively [19,23,21,22,28,32,31,6]. The classical MAB problem involves learning the optimal agent from a set of agents with a fixed but unknown reward distribution [7,28,3,30].…”

Section: Related Workmentioning

confidence: 99%

A Multi-Arm Bandit Approach To Subset Selection Under Constraints

Deva¹,

Kumar²,

Gujar³

2021

Preprint

View full text Add to dashboard Cite

We explore the class of problems where a central planner needs to select a subset of agents, each with different quality and cost. The planner wants to maximize its utility while ensuring that the average quality of the selected agents is above a certain threshold. When the agents' quality is known, we formulate our problem as an integer linear program (ILP) and propose a deterministic algorithm, namely DPSS that provides an exact solution to our ILP.We then consider the setting when the qualities of the agents are unknown. We model this as a Multi-Arm Bandit (MAB) problem and propose DPSS-UCB to learn the qualities over multiple rounds. We show that after a certain number of rounds, τ , DPSS-UCB outputs a subset of agents that satisfy the average quality constraint with a high probability. Next, we provide bounds on τ and prove that after τ rounds, the algorithm incurs a regret of O(ln T ), where T is the total number of rounds. We further illustrate the efficacy of DPSS-UCB through simulations.To overcome the computational limitations of DPSS, we propose a polynomial-time greedy algorithm, namely GSS, that provides an approximate solution to our ILP. We also compare the performance of DPSS and GSS through experiments.

show abstract

“…We begin with NAIVE(Algorithm 1), a variant of exploration separated policy, EXPSEP [21] that achieves sub-linear regret guarantee in terms of time horizon T . It is easy to see that NAIVE is fair.…”

Section: Warming Up -Naive Algorithmmentioning

confidence: 99%

Achieving Fairness in the Stochastic Multi-armed Bandit Problem

Patil

Ghalme

Nair

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

We study an interesting variant of the stochastic multi-armed bandit problem, called the Fair-SMAB problem, where each arm is required to be pulled for at least a given fraction of the total available rounds. We investigate the interplay between learning and fairness in terms of a prespecified vector denoting the fractions of guaranteed pulls. We define a fairness-aware regret, called r-Regret , that takes into account the above fairness constraints and naturally extends the conventional notion of regret. Our primary contribution is characterizing a class of Fair-SMAB algorithms by two parameters: the unfairness tolerance and the learning algorithm used as a black-box. We provide a fairness guarantee for this class that holds uniformly over time irrespective of the choice of learning algorithm. In particular, when the learning algorithm is UCB1, we show that our algorithm achieves O(ln T ) r-Regret . Finally, we evaluate the cost of fairness in terms of the conventional notion of regret.

show abstract

Mechanisms with learning for stochastic multi-armed bandit problems

Cited by 6 publications

References 11 publications

Exploring Multi-Armed Bandit (MAB) as an AI Tool for Optimising GMA-WAAM Path Planning

Exploring Multi-Armed Bandit (MAB) as an AI Tool for Optimising GMA-WAAM Path Planning

A Multi-Arm Bandit Approach To Subset Selection Under Constraints

Achieving Fairness in the Stochastic Multi-armed Bandit Problem

Contact Info

Product

Resources

About