FixedPointFinder: A Tensorflow toolbox for identifying and characterizing fixed points in recurrent neural networks

Golub, Matthew D.; Sussillo, David

doi:10.21105/joss.01003

Cited by 27 publications

(25 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Critically, RNNs were not optimized to reproduce primate behavior, only to solve specific tasks. RNNs were trained using the TensorFlow 1.14 library using standard back-propagation and adaptive hyperparameter optimization techniques (Golub and Sussillo 2018); training each RNN took one to two days on a Tesla K20 GPU.…”

Section: Methodsmentioning

confidence: 99%

The role of mental simulation in primate physical inference abilities

Rajalingham

Piccato

Jazayeri

2021

Preprint

View full text Add to dashboard Cite

Primates can richly parse sensory inputs to infer latent information, and adjust their behavior accordingly. It has been hypothesized that such flexible inferences are aided by simulations of internal models of the external world. However, evidence supporting this hypothesis has been based on behavioral models that do not emulate neural computations. Here, we test this hypothesis by directly comparing the behavior of humans and monkeys in a ball interception task to that of recurrent neural network (RNN) models with or without the capacity to “simulate” the underlying latent variables. Humans and monkeys had strikingly similar behavioral patterns suggesting common underlying neural computations. Comparison between primates and a large class of RNNs revealed that only RNNs that were optimized to simulate the position of the ball were able to accurately capture key features of the behavior such as systematic biases in the inference process. These results are consistent with the hypothesis that primates use mental simulation to make flexible inferences. Moreover, our work highlights a general strategy for using model neural systems to test computational hypotheses of higher brain function.

show abstract

Section: Methodsmentioning

confidence: 99%

The role of mental simulation in primate physical inference abilities

Rajalingham

Piccato

Jazayeri

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…One promising avenue in the analysis of trained RNNs is the application of techniques from nonlinear dynamical systems theory to interrogate the RNNs’ learned dynamical structure [14, 29, 38, 39]. The Transformer is currently disconnected from these dynamical techniques, as it lacks a recurrent structure to analyze.…”

Section: Discussionmentioning

confidence: 99%

Representation learning for neural population activity with Neural Data Transformers

Pandarinath

2021

Preprint

View full text Add to dashboard Cite

Neural population activity is theorized to reflect an underlying dynamical structure. This structure can be accurately captured using state space models with explicit dynamics, such as those based on recurrent neural networks (RNNs). However, using recurrence to explicitly model dynamics necessitates sequential processing of data, slowing real-time applications such as brain-computer interfaces. Here we introduce the Neural Data Transformer (NDT), a non-recurrent alternative. We test the NDT’s ability to capture autonomous dynamical systems by applying it to synthetic datasets with known dynamics and data from monkey motor cortex during a reaching task well-modeled by RNNs. The NDT models these datasets as well as state-of-the-art recurrent models. Further, its non-recurrence enables 3.9ms inference, well within the loop time of real-time applications and more than 6 times faster than recurrent baselines on the monkey reaching dataset. These results suggest that an explicit dynamics model is not necessary to model autonomous neural population dynamics.Codegithub.com/snel-repo/neural-data-transformers.

show abstract

“…The optimisation algorithm we have used to find fixed points is based on Sussillo and Barak [55]; see [16] for an open-source Tensorflow toolbox for finding fixed points in arbitrary RNN architectures and [25] for an alternative method to identify fixed points. The key idea is to define a scalar function whose minima correspond to fixed points of the ESN dynamics.…”

Section: Finding Fixed Points Of the Dynamicsmentioning

confidence: 99%

“…The method we propose is based on a grid of points lying in the LSS, accounting for the input action on the dynamics. By simulating the autonomous dynamics with initial conditions taken from such a grid, we are able to approximate input-driven excitability thresholds (16) and also to quantify how likely it is that the RNN uses such connections while solving the task. We define pulse difference vector (PDV) a vector containing the difference between pre-and post-input states, namely…”

Section: Determining Excitable Connections Between Attractorsmentioning

confidence: 99%

See 1 more Smart Citation

Interpreting Recurrent Neural Networks Behaviour via Excitable Network Attractors

2019

View full text Add to dashboard Cite

Introduction: Machine learning provides fundamental tools both for scientific research and for the development of technologies with significant impact on society. It provides methods that facilitate the discovery of regularities in data and that give predictions without explicit knowledge of the rules governing a system. However, a price is paid for exploiting such flexibility: machine learning methods are typically black-boxes where it is difficult to fully understand what the machine is doing or how it is operating. This poses constraints on the applicability and explainability of such methods. Methods: Our research aims to open the black-box of recurrent neural networks, an important family of neural networks used for processing sequential data. We propose a novel methodology that provides a mechanistic interpretation of behaviour when solving a computational task. Our methodology uses mathematical constructs called excitable network attractors, which are invariant sets in phase space composed of stable attractors and excitable connections between them. Results and Discussion: As the behaviour of recurrent neural networks depends both on training and on inputs to the system, we introduce an algorithm to extract network attractors directly from the trajectory of a neural network while solving tasks. Simulations conducted on a controlled benchmark task confirm the relevance of these attractors for interpreting the behaviour of recurrent neural networks, at least for tasks that involve learning a finite number of stable states and transitions between them.as reservoir computing [32,33]. Echo state networks (ESNs) [21,30] constitute an important example of reservoir computing, where a recurrent layer (called a reservoir) is composed of a large number of neurons with randomly initialised connections that are not fine-tuned via gradient-based optimisation mechanisms. The main idea behind ESNs is to exploit the rich dynamics generated by the reservoir with an output layer, the read-out that is optimised to solve a specific task. Problem statement and research hypothesisThe high-dimensional and non-linear nature of RNNs complicates interpretability of their internal dynamics, which are characterised by complex, input-dependent spatio-temporal patterns of activity [47,55]. This poses constraints on understanding the behaviour of RNNs: they are usually viewed as black-boxes from which it is hard to extract useful knowledge about their inner workings. As highlighted by recent research efforts [10,24,40], similar interpretability issues affect many other machine learning methods. Furthermore, an increasing societal need to develop accountability and explainability of decision making by AI [17] is driving the development of methodologies for explaining the behaviour of such methods.Our aim in this paper is to develop effective models that capture the essential dynamical behaviour of RNNs on computational tasks as input-driven responses of a dynamical system, while neglecting microscopic details of the RNN dynamics in phase...

show abstract

FixedPointFinder: A Tensorflow toolbox for identifying and characterizing fixed points in recurrent neural networks

Abstract: License Authors of papers retain copyright and release the work under a Creative Commons Attribution 4.0 International License (CC-BY).

Cited by 27 publications

References 5 publications

The role of mental simulation in primate physical inference abilities

The role of mental simulation in primate physical inference abilities

Representation learning for neural population activity with Neural Data Transformers

Interpreting Recurrent Neural Networks Behaviour via Excitable Network Attractors

Contact Info

Product

Resources

About