Fuzzy finite-state automata can be deterministically encoded into recurrent neural networks

Abstract

There has been an increased interest in combining fuzzy systems with neural networks because fuzzy neural systems merge the advantages of both paradigms. On the one hand, parameters in fuzzy systems have clear physical meanings and rule-based and linguistic information can be incorporated into adaptive fuzzy systems in a systematic way. On the other hand, there exist powerful algorithms for training various neural network models. However, most of the proposed combined architectures are only able to process static input-output relationships; they are not able to process temporal input sequences of arbitrary length. Fuzzy finite-state automata (FFA's) can model dynamical processes whose current state depends on the current input and previous states. Unlike in the case of deterministic finite-state automata (DFA's), FFA's are not in one particular state, rather each state is occupied to some degree defined by a membership function. Based on previous work on encoding DFA's in discrete-time second-order recurrent neural networks, we propose an algorithm that constructs an augmented recurrent neural network that encodes a FFA and recognizes a given fuzzy regular language with arbitrary accuracy. We then empirically verify the encoding methodology by correct string recognition of randomly generated FFA's. In particular, we examine how the networks' performance varies as a function of synaptic weight strengths.

Equivalence in knowledge representation: automata, recurrent neural networks, and dynamical fuzzy systems

“…Previously it has been shown that it is possible to deterministically encode FFA in recurrent neural networks by transforming any given FFA into a deterministic acceptor which assign string membership [56]. In such a deterministic encoding, only the network's classification of strings is fuzzy, whereas the representation of states is crisp.…”

“…Recent work reported in [57] addresses these issues. Instead of augmenting a second-order network with a linear output layer for computing the fuzzy string membership as suggested in [56], they chose to assign a distinct output neuron to each fuzzy string memberships occurring in the training set. Thus, the number of output neurons became equal to the number of distinct membership values…”

“…A variety of neural network implementations of FFA have been proposed [39], [40], [54], [55]. We have previously shown how fuzzy finite state automata can be mapped into recurrent neural networks with second-order weights using a crisp representation 2 of FFA states [56]. That encoding required a transformation of a fuzzy finite state automaton into a deterministic finite state automaton that computes the membership functions for strings; it is only applicable 2 A crisp mapping is one from a fuzzy to a nonfuzzy variable.…”

A New Approach to Knowledge-Based Design of Recurrent Neural Networks

“…Summarizing, the soft threshold module is realized by the second-order RNN given by (44) and (45). Note that lim ∆→0 s 2 (t) = sgn(I(t) − b), so in this case (44) implies that the module becomes a hard threshold unit.…”

“…A natural extension of the DFA-to-RNN KBD technique is based on representing the prior knowledge as an FFA and transforming this into an RNN. This is carried out using an intermediate FFA-to-DFA conversion and then applying the DFA-to-RNN method [45], or by using a direct FFA-to-RNN transformation [18]. However, FFAs may include ambiguities that make the RNN implementation difficult [18].…”