Streaming Variational Monte Carlo

Zhao, Yuan; Nassar, Josue; Jordan, Ian; Bugallo, Mónica F.; Park, Il

doi:10.1109/tpami.2022.3153225

Cited by 7 publications

(3 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Unfortunately, the analytical form of ( 9) or ( 10) is typically not tractable, especially for flexible nonlinear dynamics models. Therefore, algorithms either opt for Monte Carlo sampling (Nassar et al, 2019), variational inference (Archer et al, 2015;Pandarinath et al, 2018;Duncker et al, 2019;Zhao and Park, 2020), or hybrid (Zhao et al, 2019) approaches.…”

Section: Latent Nonlinear Continuous Dynamical Systems Modelingmentioning

confidence: 99%

“…In the low-dimensional models, the expressive power of the specific parameterization of f must be high enough to capture metastable dynamics. Radial basis function networks, Gaussian processes with squareexponential kernels, linear-nonlinear forms with hyperbolic tangent function, switching linear dynamical systems, and gated recurrent units were investigated as flexible methods of parameterizing f and shown to have sufficient expressive power in the low-dimensional regime (Zhao and Park, 2016;Duncker et al, 2019;Jordan et al, 2021;Nassar et al, 2019;Zhao et al, 2019). Due to the high flexibility of the functional form, it is important to put sufficient emphasis on simpler, more robustly generalizing functions.…”

Section: Latent Nonlinear Continuous Dynamical Systems Modelingmentioning

confidence: 99%

“…To control for the complexity of the function, various regularization methods such as penalty, simple initialization combined with early stopping strategy (Genkin and Engel, 2019), restricting the effective number of parameters, or simply imposing a prior distribution over functions are commonly used. In (Zhao et al, 2019), the authors used a sparse Gaussian processes framework to represent (a belief distribution over) f . Given a partial observation from a simulated spiking neural network with input-dependent saddle and stable fixed points, this method was able to recover the general 2D phase space through approximate Bayesian filtering.…”

Section: Latent Nonlinear Continuous Dynamical Systems Modelingmentioning

confidence: 99%

See 2 more Smart Citations

Metastable dynamics of neural circuits and networks

Brinkman,

Yan,

Maffei

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Cortical neurons emit seemingly erratic trains of action potentials, or "spikes", and neural network dynamics emerge from the coordinated spiking activity within neural circuits. These rich dynamics manifest themselves in a variety of patterns which emerge spontaneously or in response to incoming activity produced by sensory inputs. In this review, we focus on neural dynamics that is best understood as a sequence of repeated activations of a number of discrete hidden states. These transiently occupied states are termed "metastable" and have been linked to important sensory and cognitive functions. In the rodent gustatory cortex, for instance, metastable dynamics have been associated with stimulus coding, with states of expectation, and with decision making. In frontal, parietal and motor areas of macaques, metastable activity has been related to behavioral performance, choice behavior, task difficulty, and attention. In this article, we review the experimental evidence for neural metastable dynamics together with theoretical approaches to the study of metastable activity in neural circuits. These approaches include: (i) a theoretical framework based on non-equilibrium statistical physics for network dynamics; (ii) statistical approaches to extract information about metastable states from a variety of neural signals, and (iii) recent neural network approaches, informed by experimental results, to model the emergence of metastable dynamics. By discussing these topics, we aim to provide a cohesive view of how transitions between different states of activity may provide the neural underpinnings for essential functions such as perception, memory, expectation or decision making, and more generally, how the study of metastable neural activity may advance our understanding of neural circuit function in health and disease.

show abstract

Section: Latent Nonlinear Continuous Dynamical Systems Modelingmentioning

confidence: 99%

Section: Latent Nonlinear Continuous Dynamical Systems Modelingmentioning

confidence: 99%

Section: Latent Nonlinear Continuous Dynamical Systems Modelingmentioning

confidence: 99%

See 1 more Smart Citation

Metastable dynamics of neural circuits and networks

Brinkman,

Yan,

Maffei

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Dissociative and prioritized modeling of behaviorally relevant neural dynamics using recurrent neural networks

Sani,

Pesaran,

Shanechi

2024

Nat Neurosci

View full text Add to dashboard Cite

Understanding the dynamical transformation of neural activity to behavior requires new capabilities to nonlinearly model, dissociate and prioritize behaviorally relevant neural dynamics and test hypotheses about the origin of nonlinearity. We present dissociative prioritized analysis of dynamics (DPAD), a nonlinear dynamical modeling approach that enables these capabilities with a multisection neural network architecture and training approach. Analyzing cortical spiking and local field potential activity across four movement tasks, we demonstrate five use-cases. DPAD enabled more accurate neural–behavioral prediction. It identified nonlinear dynamical transformations of local field potentials that were more behavior predictive than traditional power features. Further, DPAD achieved behavior-predictive nonlinear neural dimensionality reduction. It enabled hypothesis testing regarding nonlinearities in neural–behavioral transformation, revealing that, in our datasets, nonlinearities could largely be isolated to the mapping from latent cortical dynamics to behavior. Finally, DPAD extended across continuous, intermittently sampled and categorical behaviors. DPAD provides a powerful tool for nonlinear dynamical modeling and investigation of neural–behavioral data.

show abstract

Metastable dynamics of neural circuits and networks

Brinkman

Yan

Maffei

et al. 2022

Applied Physics Reviews

View full text Add to dashboard Cite

Cortical neurons emit seemingly erratic trains of action potentials or “spikes,” and neural network dynamics emerge from the coordinated spiking activity within neural circuits. These rich dynamics manifest themselves in a variety of patterns, which emerge spontaneously or in response to incoming activity produced by sensory inputs. In this Review, we focus on neural dynamics that is best understood as a sequence of repeated activations of a number of discrete hidden states. These transiently occupied states are termed “metastable” and have been linked to important sensory and cognitive functions. In the rodent gustatory cortex, for instance, metastable dynamics have been associated with stimulus coding, with states of expectation, and with decision making. In frontal, parietal, and motor areas of macaques, metastable activity has been related to behavioral performance, choice behavior, task difficulty, and attention. In this article, we review the experimental evidence for neural metastable dynamics together with theoretical approaches to the study of metastable activity in neural circuits. These approaches include (i) a theoretical framework based on non-equilibrium statistical physics for network dynamics; (ii) statistical approaches to extract information about metastable states from a variety of neural signals; and (iii) recent neural network approaches, informed by experimental results, to model the emergence of metastable dynamics. By discussing these topics, we aim to provide a cohesive view of how transitions between different states of activity may provide the neural underpinnings for essential functions such as perception, memory, expectation, or decision making, and more generally, how the study of metastable neural activity may advance our understanding of neural circuit function in health and disease.

show abstract

Streaming Variational Monte Carlo

Cited by 7 publications

References 43 publications

Metastable dynamics of neural circuits and networks

Metastable dynamics of neural circuits and networks

Dissociative and prioritized modeling of behaviorally relevant neural dynamics using recurrent neural networks

Metastable dynamics of neural circuits and networks

Contact Info

Product

Resources

About