Theory of neuromata

Abstract

A finite automaton-the so-called neuromaton, realized by a finite discrete recurrent neural network, working in parallel computation mode, is considered. Both the size of neuromata (i.e., the number of neurons) and their descriptional complexity (i.e., the number of bits in the neuromaton representation) are studied. It is proved that a constant time delay of the neuromaton output does not play a role within a polynomial descriptional complexity. It is shown that any regular language given by a regular expression of length n is recognized by a neuromaton with ⌰(n) neurons. Further, it is proved that this network size is, in the worst case, optimal. On the other hand, generally there is not an equivalent polynomial length regular expression for a given neuromaton. Then, two specialized constructions of neural acceptors of the optimal descriptional complexity ⌰(n) for a single n-bit string recognition are described. They both require O(n 1/ 2 ) neurons and either O(n) connections with constant weights or O(n 1/ 2 ) edges with weights of the O(2 ͌n ) size. Furthermore, the concept of Hopfield languages is introduced by means of so-called Hopfield neuromata (i.e., of neural networks with symmetric weights). It is proved that the class of Hopfield languages is strictly contained in the class of regular languages. The necessary and sufficient so-called Hopfield condition stating when a regular language is a Hopfield language, is formulated. A construction of a Hopfield neuromaton is presented for a regular language satisfying the Hopfield condition. The class of Hopfield languages is shown to be closed under union, intersection, concatenation and complement, and it is not closed under iteration. Finally, the problem whether a regular language given by a neuromaton (or by a Hopfield acceptor) is nonempty, is proved to be PSPACE-complete. As a consequence, the same result for a neuromaton equivalence problem is achieved.

Stronger Separation of Analog Neuron Hierarchy by Deterministic Context-Free Languages

“…The computational power of discrete-time recurrent NNs with the saturated-linear activation function 1 depends on the descriptive complexity of their weight parameters [6,7]. NNs with integer weights, corresponding to binary-state (shortly binary) networks which employ the Heaviside activation function (with Boolean outputs 0 or 1), coincide with finite automata (FAs) recognizing regular languages [8,9,10,11,12,13]. Rational weights make the analog-state (shortly analog) NNs (with real-valued outputs in the interval [0, 1]) computationally equivalent to Turing machines (TMs) [10,14], and thus (by the real-time simulation [14]) polynomial-time computations of such networks are characterized by the fundamental complexity class P.…”

“…In addition, the units 5 and nxt from this cycle synchronize the incident neurons 6 ∈ Ṽ and out = 7 ∈ Ṽ , respectively, so that the unit 6 can be activated only at the time instants t = 3k for k > 0, by (13), while the output neuron out can fire only at the time instants τ k+1 = 3k + 1 for k ≥ 0. Hence, the result of the recognition is reported by the output neuron out as indicated in Table 1 in boldface, even for each of the four prefixes of x, the empty string ε, 1, 10, and 101, at the time instants τ 1 = 1, τ 2 = 4, τ 3 = 7, τ 4 = 10, respectively, according to (9).…”

“…for N can clearly be realized by a finite automaton, including the implementation of the finite buffer memory B. This finite automaton can further be implemented by a binary-state neural network by using the standard constructions [9,10,11,13], which is wired to the 1ANN N in order to create the 1ANN N # recognizing the language L(N # ) = L # online, as described above. In particular, the synchronization of these two networks is controlled by their input/output protocols, while the operation of N can suitably be slowed down for sufficiently large length of strings v i .…”

The Computational Power of Neural Networks and Representations of Numbers in Non-Integer Bases

“…A language L corresponds to a single-class classification (decision) problem which is solved by a neural network N , in the sense that N accepts language L containing all positive input instances of the problem that belong to the class. For the finite NNs the following input/output protocol has been used [2,6,14,15,29,30,31,32,38,40]. An input word (string) x = x 1 .…”

“…1, depends on the descriptive complexity of their weight parameters [30,38]. NNs with integer weights w ji ∈ Z, corresponding to binary-state networks with 2 s global states, coincide with finite automata which accept precisely the regular languages [2,22,40]. For example, size-optimal implementations of a given (deterministic) finite automaton with m states by using a NN with Θ( √ m) neurons have been elaborated [14,15,34].…”