Exploration in the wild

Schulz, Eric; Bhui, Rahul; Love, Bradley C.; Brier, Bastien; Todd, Michael T.; Gershman, Samuel J.

doi:10.1101/492058

Cited by 5 publications

(7 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In choosing whether to visit a new restaurant, for example, a diner might consider features such as type of cuisine, the chef's reputation, and the price of meals. Using a weighted combination of these features, they can form a generalized reward expectation for the restaurant (i.e., a reward expectation formed via feature-based generalization) even before they eat there (Schulz et al, 2019).…”

Section: Introductionmentioning

confidence: 99%

Affect-congruent attention modulates generalized reward expectations

Bennett¹,

Radulescu²,

Zorowitz

et al. 2021

Preprint

View full text Add to dashboard Cite

Positive and negative affective states are respectively associated with optimistic and pessimistic expectations regarding future reward. One mechanism that might underlie these affect-related expectation biases is attention to positive- versus negative-valence stimulus features (e.g., attending to the positive reviews of a restaurant versus its expensive price). Here we tested the effects of experimentally induced positive and negative affect on feature-based attention in 120 participants completing a compound-generalization task with eye-tracking. We found that participants' reward expectations for novel compound stimuli were modulated by the affect induction in an affect-congruent way: positive affect increased reward expectations for compounds, whereas negative affect decreased reward expectations. Computational modelling and eye-tracking analyses each revealed that these effects were driven by affect-congruent changes in participants' allocation of attention to high- versus low-value features of compound stimuli. These results provide mechanistic insight into a process by which affect produces biases in generalized reward expectations.

show abstract

Section: Introductionmentioning

confidence: 99%

Affect-congruent attention modulates generalized reward expectations

Bennett¹,

Radulescu²,

Zorowitz

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Exploration in the lab is often described as probabilistic ( Calhoun et al, 2014 ; Song et al, 2019 ; Gershman, 2018 ; Schulz et al, 2018 ). By definition probabilistic models do not make exact predictions of behavior, only statistical ones.…”

Section: Discussionmentioning

confidence: 99%

“…The choice between exploring and exploiting is a kind of decision that is faced routinely by learners of all kinds, including foraging bees, business organizations, humans, worms, monkeys, rodents, birds, children, and computer algorithms ( Gupta et al, 2006 ; Sutton and Barto, 2018 ; Woodgate et al, 2017 ; Lee et al, 2011 ; Schulz et al, 2018 ; Calhoun et al, 2014 ; Wang and Hayden, 2019 ; Sumner et al, 2019 ; Auersperg, 2015 ). But is reward really fundamental to it?…”

Section: Figurementioning

confidence: 99%

See 1 more Smart Citation

Embracing curiosity eliminates the exploration-exploitation dilemma

Ej¹,

Verstynen²

2019

Preprint

View full text Add to dashboard Cite

For all animals the decision to explore comes with a risk of getting less. For example, a foraging bee might find less nectar, or hunting hawk less prey. This loss is often formalized as regret. It's been mathematically proven that exploring an uncertain world with a specific goal always has some regret. This is why exploration-exploitation can be a dilemma. Given this proof we wondered if the common advice to "focus on learning and not the goal" might have mathematical merit. So we re-imagined exploration in the dilemma as an open ended search for any new information. We then developed a new minimal description of information value, which generalizes existing ideas like curiosity, novelty and information gain. We use this description to model the dilemma as a competition between strategies that maximize reward and information independently. Here we prove this competition has a no regret solution. When we study this solution in simulation -using classic bandit tasks -it outperforms standard approaches, especially when rewards are sparse.

show abstract

“…Participants may not easily be able to turn off the exploitation part of their sampling strategy as they normally encounter a mix of exploration and exploitation problems in real life (Schulz, Bhui, et al, 2019).…”

Section: Discussionmentioning

confidence: 99%

Learning functions actively

Jones¹,

Schulz²,

Meder³

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

How do people actively learn functional rules, i.e. a mapping of continuous inputs onto a continuous output? We investigate information search behavior in a multiple-featurefunction learning task in which participants either actively select or passively receive observations. We find that participants benefit from actively selecting information, inparticular in their function extrapolation performance. By introducing and comparing different models of active function learning, we find that participants are best described by a non-parametric function learning model that learns about both the underlying function and inputs that are likely to produce high outputs. These results enrich our understandingof active function learning in complex domains.

show abstract

Exploration in the wild

Cited by 5 publications

References 32 publications

Affect-congruent attention modulates generalized reward expectations

Affect-congruent attention modulates generalized reward expectations

Embracing curiosity eliminates the exploration-exploitation dilemma

Learning functions actively

Contact Info

Product

Resources

About