Interface Design Optimization as a Multi-Armed Bandit Problem

Lomas, Derek; Forlizzi, Jodi; Poonwala, Nikhil; Patel, Nirmal; Shodhan, Sharan; Patel, Kishan; Koedinger, Ken; Brunskill, Emma

doi:10.1145/2858036.2858425

Cited by 39 publications

(25 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We implemented Sarsa, which is a standard algorithm to learn how to act in many different environment state, i.e., for each given parameter configuration [97]. It differs from multi-armed bandits, which learns how to act in one unique environment state [68]. Importantly, as evoked in Section 1, Sarsa was designed to learn one optimal behaviour in relation to the goal of a task.…”

Section: Methodsmentioning

confidence: 99%

“…In this sense, interactive reinforcement learning relies on small, user-specific data sets, which contrasts with the large, crowdsourced data sets used in creative applications in semantic editing [25,62,107]. Lastly, interactive approaches to reinforcement learning focuses on exploring agent actions based on human feedback on actions, which contrasts with the focus on optimising one parametric state based on user feedback over states-as used in Bayesian Optimisation [13,67] or multi-armed bandits [68].…”

Section: Interactive Reinforcement Learningmentioning

confidence: 99%

“…Interactive reinforcement learning has been recently applied in HCI [84], with promising applications in exploratory search [10,44] and adaptive environments [40,80]. Integrating user feedback in reinforcement learning algorithms is computationally feasible [94], helps agents learn better [57], can make data-driven design more accessible [68], and holds potential for rich human-computer collaboration [95]. Applications in Human-Robot Interaction informed on how humans may give feedback to learning agents [98], and showed potential for enabling human-robot co-creativity [36].…”

Section: Interactive Reinforcement Learningmentioning

confidence: 99%

See 2 more Smart Citations

Designing Deep Reinforcement Learning for Human Parameter Exploration

Scurto

Kerrebroeck

Caramiaux

et al. 2021

ACM Trans. Comput.-Hum. Interact.

View full text Add to dashboard Cite

Software tools for generating digital sound often present users with high-dimensional, parametric interfaces, that may not facilitate exploration of diverse sound designs. In this article, we propose to investigate artificial agents using deep reinforcement learning to explore parameter spaces in partnership with users for sound design. We describe a series of user-centred studies to probe the creative benefits of these agents and adapting their design to exploration. Preliminary studies observing users’ exploration strategies with parametric interfaces and testing different agent exploration behaviours led to the design of a fully-functioning prototype, called Co-Explorer, that we evaluated in a workshop with professional sound designers. We found that the Co-Explorer enables a novel creative workflow centred on human–machine partnership, which has been positively received by practitioners. We also highlight varied user exploration behaviours throughout partnering with our system. Finally, we frame design guidelines for enabling such co-exploration workflow in creative digital applications.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Interactive Reinforcement Learningmentioning

confidence: 99%

Section: Interactive Reinforcement Learningmentioning

confidence: 99%

See 1 more Smart Citation

Designing Deep Reinforcement Learning for Human Parameter Exploration

Scurto

Kerrebroeck

Caramiaux

et al. 2021

ACM Trans. Comput.-Hum. Interact.

View full text Add to dashboard Cite

show abstract

“…Online interface refinement, the category in which this paper falls, describes methods which actively change the interface based on some objective during or between interactions. This approach is readily applied in games where an optimal performance or engagement level might be achieved through game feature refinement [7,14,15]. Similarly, BIGnav [12] probabilistically fused inputs and prior information about locations on a map to improve navigation performance.…”

Section: Related Workmentioning

confidence: 99%

Crowdsourcing Interface Feature Design with Bayesian Optimization

Dudley

Jacques

Kristensson

2019

Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

Designing novel interfaces is challenging. Designers typically rely on experience or subjective judgment in the absence of analytical or objective means for selecting interface parameters. We demonstrate Bayesian optimization as an efficient tool for objective interface feature refinement. Specifically, we show that crowdsourcing paired with Bayesian optimization can rapidly and effectively assist interface design across diverse deployment environments. Experiment 1 evaluates the approach on a familiar 2D interface design problem: a map search and review use case. Adding a degree of complexity, Experiment 2 extends Experiment 1 by switching the deployment environment to mobile-based virtual reality. The approach is then demonstrated as a case study for a fundamentally new and unfamiliar interaction design problem: web-based augmented reality. Finally, we show how the model generated as an outcome of the refinement process can be used for user simulation and queried to deliver various design insights. CCS CONCEPTS • Human-centered computing → Systems and tools for interaction design.

show abstract

“…Lan and Baraniuk [8] used sparse factor analysis with bandits to identify sequences of educational content that could maximize students performance on subsequent assessments. Lomas et al [9] showed how bandits can be used to search a large space of design decisions in creating educational games. Williams et al [18] used Thompson Sampling to identify highly rated explanations for how to solve Math problems, and chose priors that assumed that every explanation was equally rated.…”

Section: Related Workmentioning

confidence: 99%

Combining Difficulty Ranking with Multi-Armed Bandits to Sequence Educational Content

Segal

David

Williams

et al. 2018

Lecture Notes in Computer Science

View full text Add to dashboard Cite

As e-learning systems become more prevalent, there is a growing need for them to accommodate individual differences between students. This paper addresses the problem of how to personalize educational content to students in order to maximize their learning gains over time. We present a new computational approach to this problem called MAPLE (Multi-Armed Bandits based Personalization for Learning Environments) that combines difficulty ranking with multiarmed bandits. Given a set of target questions MAPLE estimates the expected learning gains for each question and uses an exploration-exploitation strategy to choose the next question to pose to the student. It maintains a personalized ranking over the difficulties of question in the target set which is used in two ways: First, to obtain initial estimates over the learning gains for the set of questions. Second, to update the estimates over time based on the students responses. We show in simulations that MAPLE was able to improve students' learning gains compared to approaches that sequence questions in increasing level of difficulty, or rely on content experts. When implemented in a live e-learning system in the wild, MAPLE showed promising results. This work demonstrates the efficacy of using stochastic approaches to the sequencing problem when augmented with information about question difficulty.

show abstract

Interface Design Optimization as a Multi-Armed Bandit Problem

Cited by 39 publications

References 27 publications

Designing Deep Reinforcement Learning for Human Parameter Exploration

Designing Deep Reinforcement Learning for Human Parameter Exploration

Crowdsourcing Interface Feature Design with Bayesian Optimization

Combining Difficulty Ranking with Multi-Armed Bandits to Sequence Educational Content

Contact Info

Product

Resources

About