Neuromodulated attention and goal-driven perception in uncertain domains

Abstract

In uncertain domains, the goals are often unknown and need to be predicted by the organism or system. In this paper, contrastive excitation backprop (c-EB) was used in a goal-driven perception task with pairs of noisy MNIST digits, where the system had to increase attention to one of the two digits corresponding to a goal (i.e., even, odd, low value, or high value) and decrease attention to the distractor digit or noisy background pixels. Because the valid goal was unknown, an online learning model based on the cholinergic and noradrenergic neuromodulatory systems was used to predict a noisy goal (expected uncertainty) and re-adapt when the goal changed (unexpected uncertainty). This neurobiologically plausible model demonstrates how neuromodulatory systems can predict goals in uncertain domains and how attentional mechanisms can enhance the perception of that goal.Artificial top-down attentional systems tend to respond to sensory inputs similarly regardless of context and goals. However, biological systems select relevant information to guide behavior in the face of noisy and unreliable signals, as well as rapidly adapt to unforeseen situations. Goal-driven perception treats the same situation differently based on context and causes attention to be directed to goal-relevant inputs. Oftentimes, these goals are unknown and must be learned through experience. Moreover, these goals or contexts can shift without warning. Goal-driven perception helps prevent over-emphasizing less relevant stimuli and focus instead on critical stimuli that require an immediate response.Neuromodulators are important contributors for attention and goal-driven perception. In particular, the cholinergic system drives bottom-up, stimulus-driven attention, as well as top-down, goal-directed attention [1]. Furthermore, the cholinergic system increases attention to task-relevant stimuli, while decreasing attention to the distractions [2,3]. This is a similar idea to contrastive Excitatory Backpropagation (c-EB) where a top-down excitation mask increments attention to the target features and an inhibitory mask decrements attention to distractors [4]. The noradrenergic system responds to surprises or large violations of priors. When the noradrenergic system responds phasically, where the neural activity rapidly and transiently increases, it causes a network reset (e.g., re-initializing weights or connections) that allows rapid adaptation and relearning under novel conditions [5,6].We modified a c-EB network for use in a goal-driven perception task, where the system had to increase attention to the intended goal object and decrease attention to the distractor. Specifically, we presented to the network pairs of noisy MNIST digits. One goal class was to attend to the digit based on its parity (i.e., even or odd goal), and the other goal class was to attend based on the magnitude of the digit (i.e., low-value or high-value goal). In addition, we added a neuromodulation model to the head of the network architecture that regulated goal selection. Simi...

CogSense: A Cognitively Inspired Framework for Perception Adaptation

“…In turn, experimental evidence also supports mid-level cognitive adaptation in the level of alertness and orienting behavior to reacquire new statistics of the world [13]. Indeed, such models have been used in artificial neural networks for adapting behavior in robots [14,15].…”

Design Principles for Neurorobotics

“…The neuromodulatory system also regulates attention allocation and response to unexpected events. Using the Toyota Human Support Robot (Yamamoto et al, 2018 ), the influence of the cholinergic (ACh) system and noradrenergic (NE) systems on goal-directed perception was studied in an action-based attention task (Zou et al, 2020 ). In this experiment, a robot was required to attend to goal-related objects (the ACh system) and adjust to the change of goals in an uncertain domain (the NE system).…”

Context Meta-Reinforcement Learning via Neuromodulation

“…Several studies have drawn inspiration from neuromodulation and applied it to gradient-based RL (Xing et al, 2020;Miconi et al, 2020) and neuroevolutionary RL (Soltoggio et al, 2007(Soltoggio et al, , 2008Velez and Clune, 2017) for dynamic task settings. In broader machine learning, neuromodulation has been applied to goal-driven perception (Zou et al, 2020), and also in continual learning setting (Beaulieu et al, 2020) where it was combined with meta-learning to sequentially learn a number of classification tasks without catastrophic forgetting. The neuromodulators used in these studies have different designs or functions: plasticity gating (Soltoggio et al, 2008;Miconi et al, 2020), activation gating (Beaulieu et al, 2020), direct action modification in a policy (Xing et al, 2020).…”