A Spotlight on Reward

Akaishi, Rei; Hayden, Benjamin Y.

doi:10.1016/j.neuron.2016.06.008

Cited by 8 publications

(7 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(Lower Left) The optimization process in machine learning and animals involves some sort of gradient ascent (or descent) where the state or agent follows the local gradient. This space can be considered continuous—Such as with “odor-scapes” of smell gradients—Or discrete—With different trees representing different patches, similar to multi-alternative choice environments (Upper right image adapted and modified with permission from Akaishi and Hayden, 2016, Neuron, Cell Press [ 35 ]).…”

Section: Figurementioning

confidence: 99%

“…We believe that both fields stand to gain from a common understanding of this problem. To do this, we first focus on experimental and theoretical investigations aimed at understanding multiscale processing in the brain and biological systems more generally [ 35 , 56 , 57 , 58 , 59 , 60 , 61 , 62 , 63 , 64 , 65 , 66 ]. We start with temporal processing and then move to spatial processing, before tackling the neural underpinnings.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Multiscale Computation and Dynamic Attention in Biological and Artificial Intelligence

2020

Self Cite

View full text Add to dashboard Cite

Biological and artificial intelligence (AI) are often defined by their capacity to achieve a hierarchy of short-term and long-term goals that require incorporating information over time and space at both local and global scales. More advanced forms of this capacity involve the adaptive modulation of integration across scales, which resolve computational inefficiency and explore-exploit dilemmas at the same time. Research in neuroscience and AI have both made progress towards understanding architectures that achieve this. Insight into biological computations come from phenomena such as decision inertia, habit formation, information search, risky choices and foraging. Across these domains, the brain is equipped with mechanisms (such as the dorsal anterior cingulate and dorsolateral prefrontal cortex) that can represent and modulate across scales, both with top-down control processes and by local to global consolidation as information progresses from sensory to prefrontal areas. Paralleling these biological architectures, progress in AI is marked by innovations in dynamic multiscale modulation, moving from recurrent and convolutional neural networks—with fixed scalings—to attention, transformers, dynamic convolutions, and consciousness priors—which modulate scale to input and increase scale breadth. The use and development of these multiscale innovations in robotic agents, game AI, and natural language processing (NLP) are pushing the boundaries of AI achievements. By juxtaposing biological and artificial intelligence, the present work underscores the critical importance of multiscale processing to general intelligence, as well as highlighting innovations and differences between the future of biological and artificial intelligence.

show abstract

Section: Figurementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Multiscale Computation and Dynamic Attention in Biological and Artificial Intelligence

2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…Monkeys performing the decision task with advanced information for reward also showed more preference for the information of the greater reward 10 . These results can be interpreted that attention/information seeking is an intrinsic part of a goal-directed operation in foraging behavior so that the properties of the goal itself influences the information seeking behavior 38 . Thus, the current results of sequential gaze patterns are also consistent with this emergent framework of attention and goal-directed behavior.…”

Section: Discussionmentioning

confidence: 81%

“…These influences of gaze on the contextual cues can be interpreted as a pure manifestation of information sampling as reported in the previous studies [28][29][30] . However, the gaze behavior and information seeking can be interpreted as a reflection of simulation process 9,10,31,38 . Specifically, these results can indicate that the simulation of association sequence made it easier to choose the associated target in that sequence and difficult to choose the target not in the simulated sequence.…”

Section: Influences Of the Gaze To Contextual Cues On Target Selectionmentioning

confidence: 99%

Information seeking and simulation: Roles of attention in guiding a goal-directed behavior

Akaishi

Hoshi

2017

Preprint

Self Cite

View full text Add to dashboard Cite

We usually actively seek out the information we need. However, it is still debated whether information seeking in decision situation is a purposeful behavior or a random process. We investigated this issue using the decision task involving multiple goal-directed event sequences, in which a contextual cue specifies an associated target and touch to the target delivers the reward. We found that the gaze followed the sequence of contextual cue to the associated target, which was eventually chosen. This fixation sequence from contextual cue to the associated target could be observed even when there were multiple goals and when the focus was shifted from one goal to another. To causally investigate the effects of sequential simulation, we directly manipulated the processing of the contextual cues and found its influence on the final choice of target. Furthermore, past episodes of the sequences influenced both final choices of targets and initial gaze to contextual cues. We interpret the results as suggesting that the internal process of simulating goal-directed event sequence drives information-seeking behavior such as attention/gaze in decision situations.. CC-BY 4.0 International license peer-reviewed) is the author/funder. It is made available under a The copyright holder for this preprint (which was not . http://dx.doi.org/10.1101/104091 doi: bioRxiv preprint first posted online 3 IntroductionWhen we consume information, we are rarely passive recipient. Rather, we actually seek out the information we desired. For example, we click and move to the specific webpage with some expectation of the content and interpret the information with that prior conception. Thus, when we seek new information, somewhat paradoxically, we already have some knowledge of its content. However, it is still debated how and why the prior knowledge can influence the selection of the information to be processed.Such decision to choose and process specific information has been studied in the research of attention 1,2 . There have been proposals about the factors guiding attention. One line of thought is purely informational: information seeking behavior is to satisfy the pure need for information, which in turn determined by the gap between current level of the knowledge and the information available in the environment 3,4 . Another line of thought is that information seeking behavior is driven by valence or reward-related aspects of the information 1,4-8 . These two types of the influences on information seeking behavior are treated separately 1,4,5 . However, recent studies suggest that there are interactions between the two factors 4,9,10 . There can be some mechanism mediating the interaction between the informational and reward-related factors or there can be an intrinsic link between the two factors.Attention is also known to guide the direction of our action to achieve a goal. For example, we always pay attention to the direction we are heading when we are driving a car 11,12 .If you intend to turn right, your gaze scans th...

show abstract

“…To achieve a better understanding of how stress and cognition interact to modulate the reward processes, it is necessary to determine how each of these factors per se influences motivation and hedonic experience. Previous research has focused on the complex relationship between cognition, motivation, and hedonic capacities (Akaishi & Hayden, ; Esterman et al, ; O'Connor, Rossiter, Yücel, Lubman, & Hester, ; Rothkirch, Schmack, Deserno, Darmohray, & Sterzer, ). A large body of research evidenced the effort‐discounting effect on reward valuation, so that the effort exerted to obtain a desired reward decreases as effort cost increases (for a review, see Kurniawan, ).…”

Section: Introductionmentioning

confidence: 99%

Striatal responsiveness to reward under threat‐of‐shock and working memory load: A preliminary study

et al. 2019

View full text Add to dashboard Cite

Introduction Reward and stress are important determinants of motivated behaviors. Striatal regions play a crucial role in both motivation and hedonic processes. So far, little is known on how cognitive effort interacts with stress to modulate reward processes. This study examines how cognitive effort (load) interacts with an unpredictable acute stressor (threat‐of‐shock) to modulate motivational and hedonic processes in healthy adults. Materials and Methods A reward task, involving stress with unpredictable mild electric shocks, was conducted in 23 healthy adults aged 20–37 (mean age: 24.7 ± 0.9; 14 females) during functional magnetic resonance imaging (fMRI). Manipulation included the use of (a) monetary reward for reinforcement, (b) threat‐of‐shock as the stressor, and (c) a spatial working memory task with two levels of difficulty (low and high load) for cognitive load. Reward‐related activation was investigated in a priori three regions of interest, the nucleus accumbens (NAcc), caudate nucleus, and putamen. Results During anticipation, threat‐of‐shock or cognitive load did not affect striatal responsiveness to reward. Anticipated reward increased activation in the ventral and dorsal striatum. During feedback delivery, both threat‐of‐shock and cognitive effort modulated striatal activation. Higher working memory load blunted NAcc responsiveness to reward delivery, while stress strengthened caudate nucleus reactivity regardless reinforcement or load. Conclusions These findings provide initial evidence that both stress and cognitive load modulate striatal responsiveness during feedback delivery but not during anticipation in healthy adults. Of clinical importance, sustained stress exposure might go along with dysregulated arousal, increasing therefore the risk for the development of maladaptive incentive‐triggered motivation. This study brings new insight that might help to build a framework to understand common stress‐related disorders, given that these psychiatric disorders involve disturbances of the reward system, cognitive deficits, and abnormal stress reactivity.

show abstract

A Spotlight on Reward

Cited by 8 publications

References 12 publications

Multiscale Computation and Dynamic Attention in Biological and Artificial Intelligence

Multiscale Computation and Dynamic Attention in Biological and Artificial Intelligence

Information seeking and simulation: Roles of attention in guiding a goal-directed behavior

Striatal responsiveness to reward under threat‐of‐shock and working memory load: A preliminary study

Contact Info

Product

Resources

About