Myopic control of neural dynamics

Abstract

Manipulating the dynamics of neural systems through targeted stimulation is a frontier of research and clinical neuroscience; however, the control schemes considered for neural systems are mismatched for the unique needs of manipulating neural dynamics. An appropriate control method should respect the variability in neural systems, incorporating moment to moment "input" to the neural dynamics and behaving based on the current neural state, irrespective of the past trajectory. We propose such a controller under a nonlinear state-space feedback framework that steers one dynamical system to function as through it were another dynamical system entirely. This "myopic" controller is formulated through a novel variant of a model reference control cost that manipulates dynamics in a step-wise manner, omitting the need to pre-calculate a rigid and computationally costly neural feedback control solution. To demonstrate the breadth of this control's utility, two examples with distinctly different applications in neuroscience are studied. First an unhealthy motor-like system containing an unwanted beta-oscillation spiral attractor is controlled to function as a healthy motor system, a relevant clinical example for neurological disorders. Second, we show the myopic control's utility to probe the causal dynamics of cognitive processes by transforming a winner-take-all decision-making system to operate as a robust neural integrator of evidence.Advances in recording technology are making it possible to gain real-time access to neural dynamics at different length and time scales [bra, 2014[bra, , Jun et al., 2017, allowing us to consider the structure of the brain's operation in ways that were previously inaccessible. Central to that understanding of neural dynamics is the widely-held belief that dynamical systems underlie all of the core operations of neural systems [Breakspear, 2017, Fairhall and 1 peer-reviewed)

Embracing curiosity eliminates the exploration-exploitation dilemma

“…We feel this approach has several advantages. It is myopic and therefore can optimally handle nonlinear changes in either the reward function or environmental dynamics (67). While in stationary settings, its regret is bounded (68).…”

“…We expect it to vary substantially before contracting to approach 0. It's best possible value is well approximated then by only looking at the maximum value for the last time step, making our optimization myopic, that is based on only last values encountered (Hocker and Park, 2019).…”

“…This is a highly nonlinear dynamical system (Strogatz, 1994). In such systems, myopic control-using only the most recent value to predict the next value-is known to be an robust and efficient form of control (Hocker and Park, 2019). We therefore assume that last value is the best predictor of the next value, and use this assumption along with Theorem 5 to complete a trivial proof that Eq.…”

Variational Online Learning of Neural Dynamics

“…Our primary applied aim is real-time neural interfaces where a population of neurons are recorded while a low-dimensional stimulation is delivered (Newman et al, 2015 ; El Hady, 2016 ; Hocker and Park, 2019 ). State-space modeling of such neural time series have been successful in describing population dynamics (Macke et al, 2011 ; Zhao and Park, 2017 ).…”

Birhythmic Analog Circuit Maze: A Nonlinear Neurostimulation Testbed

“…However, open-loop strategies are likely to be insufficient for the general induction of attractor transitions that manifest complex nonlinear dynamics and non-trivial stimulus induced perturbations. For example, high-amplitude low-frequency signals, such as those dominant in disorders of consciousness [ 27 ], suggest the existence of strong attractor dynamics which may require a sophisticated feedback control system to transition out [ 29 ]. This sets the stage for the next-generation closed-loop neural stimulators that can intelligently learn to induce attractor transitions.…”