Complex dynamics and the structure of small neural networks

Abstract

The discrete-time dynamics of small neural networks is studied empirically, with emphasis laid on non-trivial bifurcation scenarios. For particular two- and three-neuron networks interesting dynamical properties like periodic, quasi-periodic and chaotic attractors are observed, many of them co-existing for one and the same set of parameters. An appropriate equivalence class of networks is defined, describing them as parametrized dynamical systems with identical dynamical capacities. Combined symmetries in phase space and parameter space are shown to generate different representatives of such a class. Moreover, conditions on the connectivity structure are suggested, which guarantee the existence of complex dynamics for a considered equivalence class of network configurations.

The Emergence of Communication by Evolving Dynamical Systems

“…By means of an odd loop with over-critical synaptic weights 2 , the sound generating output neuron O5 is connected with a hidden neuron (H1). This loop acts as a switchable oscillator [11] depending on the value of I4, the floor sensor input. I4 is equal to -1.0 as long as the robot is moving on white ground.…”

“…The output o i = f (a i ) of a unit i is given by a sigmoid transfer function, here by f := tanh (i.e., o i ∈ (−1, 1)). Although this neuron model is rather simple, already small recurrent networks of this type can generate complex dynamics, such as periodic, quasi-periodic, or even chaotic attractors [11]. For the evolution of these dynamical systems we used an implementation (see [18] for details) of the evolutionary algorithm EN S 3 [19].…”

“…As artificial brain structures we utilize recurrent neural networks (RNNs) which can be described as parameterized dynamical systems [11]. We distinguish two types of parameters.…”

SO(2)-Networks as Neural Oscillators

“…Aspects of discrete-time dynamics of two neuron networks with recurrent connectivity have been studied for a long time, e. g. [5], [8], [1], [2], [6]. This is because they are the simplest neural networks having non-trivial dynamical properties: for certain parameter domains one finds not only stationary attractors but oscillations of various periodicity, quasi-periodic and chaotic dynamics.…”

“…There are various hysteresis phenomena observable, and different bifurcation scenarios involved. Most of these complex dynamical properties can be observed for recurrently coupled inhibitory and excitatory neurons with appropriate selfconnections [6].…”