Learning automata from ordered examples

Abstract

Connectionist learning models have had considerable empirical success, but it is hard to characterize exactly what they learn. The learning of finite-state languages (FSL) from example strings is a domain which has been extensively studied and might provide an opportunity to help understand connectionist learning. A major problem is that traditional FSL learning assumes the storage of all examples and thus violates connectionist principles. This paper presents a provably correct algorithm for inferring any minimum-state deterministic finite-state automata (FSA) from a complete ordered sample using limited total storage and without storing example strings. The algorithm is an iterative strategy that uses at each stage a current encoding of the data considered so far, and one single sample string. One of the crucial advantages of our algorithm is that the total amount of space, used in the course oflearning, for encoding any finite prefix of the sample is polynomial in the size of the inferred minimum state deterministic FSA. The algorithm is also relatively efficient in time and has been implemented. More importantly, there is a connectionist version of the algorithm that preserves these properties. The connectionist version requires much more structure than the usual models and has not yet been implemented. But it does significantly extend the scope of connectionist learning systems and helps relate them to other paradigms. We also show that no machine with finite working storage can identify iteratively the FSL from arbitrary presentations.The Xerox Corporation University Grants Program provided equipment used in the preparation of this paper.

Ignoring data may be the only way to learn efficiently

“…In Wiehagen (1976) CONS C IT LIM has been proved. Recent papers give new evidence for the power of iterative learning (Porat and Feldman 1988, Lange and Wiehagen 1991, Lange and Zeugmann 1992.…”

The Kernel Adaptive Autoregressive-Moving-Average Algorithm

“…This is a more difficult problem, since it has been shown that learning is enhanced using lexicographic order in string presentation [56]. In practice, one cannot expect the training samples to be a complete ordered set, especially for unknown grammars.…”

Inferring state-based behavior models

“…Several are based on hypotheses that are not satisfied in the application domains, e.g., the presence of negative samples [5], the availability of teachers [12] or information on the order of the traces [13]. We developed our algorithm by extending the k-Tail algorithm and its many variants that work well on positive samples only [3,4,14] The GK-Tail algorithm proposed in this paper generates finite state automata augmented with parameters (FSAP) from sets of program traces in three steps.…”