Bandit Algorithms for Precision Medicine

Lu, Yangyi; Xu, Ziping; Tewari, Ambuj

doi:10.48550/arxiv.2108.04782

Cited by 3 publications

(5 citation statements)

References 74 publications

(97 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This setting has been extended to CBs (Kazerouni et al, 2017;Garcelon et al, 2020b;Lin et al, 2022) and Reinforcement learning (Garcelon et al, 2020a). For a more careful discussion on the above settings we refer the reader to (Lu et al, 2021).…”

Section: A Related Workmentioning

confidence: 99%

Leveraging User-Triggered Supervision in Contextual Bandits

Agarwal¹,

Gentile²,

Marinov³

2023

Preprint

View full text Add to dashboard Cite

We study contextual bandit (CB) problems, where the user can sometimes respond with the best action in a given context. Such an interaction arises, for example, in text prediction or autocompletion settings, where a poor suggestion is simply ignored and the user enters the desired text instead. Crucially, this extra feedback is usertriggered on only a subset of the contexts. We develop a new framework to leverage such signals, while being robust to their biased nature. We also augment standard CB algorithms to leverage the signal, and show improved regret guarantees for the resulting algorithms under a variety of conditions on the helpfulness of and bias inherent in this feedback.

show abstract

Section: A Related Workmentioning

confidence: 99%

Leveraging User-Triggered Supervision in Contextual Bandits

Agarwal¹,

Gentile²,

Marinov³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Literature shows that the lack of time is the main reason for not performing physical activities [24]. Studies indicate that even 30 minutes of movement per week can improve health [24].…”

Section: Exercise Organizationmentioning

confidence: 99%

“…Literature shows that the lack of time is the main reason for not performing physical activities [24]. Studies indicate that even 30 minutes of movement per week can improve health [24]. Hence, since we had a healthy younger population cohort, we chose 15 and 20 minutes as the baseline for level 1 and 2 groups, respectively.…”

Section: Exercise Organizationmentioning

confidence: 99%

See 1 more Smart Citation

PERFECT: Personalized Exercise Recommendation Framework and architECTure

Mehrabadi,

Khatibi,

Jimah

et al. 2023

Preprint

View full text Add to dashboard Cite

Background: There are indisputable health benefits to physical activity (PA). By collecting and displaying individual exercise behaviors via wearable trackers, the Internet of Things (IoT) and mobile health (mHealth) have made it possible to correlate users' physiological data and daily activity information with their fitness requirements. Objective: This study aimed to recommend personalized exercise to non-pregnant subjects to increase their physical activity level. Methods: We developed smartphone and smartwatch applications to collect, monitor, and recommend exercises using a contextual multi-arm bandit framework. Twenty female college students were recruited to test this mHealth exercise program. Results: Our findings indicated an increase in daily exercise duration (P < .001), with average satisfaction scores for the walking and recommendation system components of 4.31 (0.60) and 3.69 (0.95), respectively, on a scale of 1 to 5. In addition, participants' confidence in their capacity to complete the suggested walking exercises safely and the study's ability to satisfy their needs for physical activity both received average scores of over 4. Conclusions: A new era of mHealth systems has been ushered in by developments in the Internet of Things and wearable devices. Personalization of physical activity recommendations using such wearables has the potential to improve user engagement and performance. In this paper, we presented an exercise recommendation system based on reinforcement learning that uses biomarkers and the user's context to recommend a unique walking exercise that enhances the user's aerobic capacity.

show abstract

“…Sequential decision-making is one of the essential components of modern artificial intelligence that considers the dynamics of the real world. By maintaining the trade-off between exploration and exploitation based on historical information, the bandit algorithms aim to maximize the cumulative outcome of interest and thus are popular in dynamic decision optimization with a wide variety of applications, such as precision medicine (Lu et al, 2021) and dynamic pricing (Turvey, 2017). There is a large literature on bandit optimization established during recent decades (see the reviews in Sutton and Barto, 2018;Lattimore and Szepesvári, 2020).…”

Section: Introductionmentioning

confidence: 99%

Doubly Robust Interval Estimation for Optimal Policy Evaluation in Online Learning

Cai

Song

2021

Preprint

View full text Add to dashboard Cite

Evaluating the performance of an ongoing policy plays a vital role in many areas such as medicine and economics, to provide crucial instruction on the early-stop of the online experiment and timely feedback from the environment. Policy evaluation in online learning thus attracts increasing attention by inferring the mean outcome of the optimal policy (i.e., the value) in real-time. Yet, such a problem is particularly challenging due to the dependent data generated in the online environment, the unknown optimal policy, and the complex exploration and exploitation trade-off in the adaptive experiment. In this paper, we aim to overcome these difficulties in policy evaluation for online learning. We explicitly derive the probability of exploration that quantifies the probability of exploring the non-optimal actions under commonly used bandit algorithms. We use this probability to conduct valid inference on the online conditional mean estimator under each action and develop the doubly robust interval estimation (DREAM) method to infer the value under the estimated optimal policy in online learning. The proposed value estimator provides double protection on the consistency and is asymptotically normal with a Wald-type confidence interval provided. Extensive simulations and real data applications are conducted to demonstrate the empirical validity of the proposed DREAM method.

show abstract

Bandit Algorithms for Precision Medicine

Cited by 3 publications

References 74 publications

Leveraging User-Triggered Supervision in Contextual Bandits

Leveraging User-Triggered Supervision in Contextual Bandits

PERFECT: Personalized Exercise Recommendation Framework and architECTure

Doubly Robust Interval Estimation for Optimal Policy Evaluation in Online Learning

Contact Info

Product

Resources

About