System Identification, Approximation and Complexity

Abstract

This paper is concerned with establishing broadly based system theoretic foundations and practical techniques for the problem of system identification that are rigorous, intuitively clear and conceptually powerful. A general formulation is first given in which two order relations are postulated on a class of models: a constant one of complexity and a variable one of approximation induced by an observed behaviour. An admissible model is such that any less complex model is a worse approximation. The general problem of identification is that of finding the admissible subspace of models induced by a given behaviour. It is proved under very general assumptions that. ifdeterministic models are required, then nearly all behaviours require models of nearly maximum complexity. A general theory of approximation between models and behaviour is then developed based on subjective probability concepts and semantic information theory. The role of structural constraints such as causality, locality, finite memory. etc., are then discussed as rules a/the game. These concepts and results are applied to the specific problem of stochastic automaton. or grammar. inference. Computational results arc given to demonstrate that the theory is complete and fully operational. Finally the formulation of identification proposed in this paper is analysed in terms of Klir's epistemological hierarchy ami both arc discussed in terms of the rich philosophical literature on the acquisition of knowledge.I NDEX TERMS Systems identification, systems approximation, systems complexity, admissible model, epistemological hierarchy, grammatical inference, logical probability.

Progress in General Systems Research

“…In retrospect I feel that we had become trapped by the statistician's view of probability theory (which is that taught to engineers) and that the very much more fruitful logical and algebraic foundations were insufficiently known, being neglected both in theory and practice. Terrence Fine's (1973) book on Theories of Probability does much to bring these foundations together and a wide range of technical and philosophical results may be welded into a coherent system (Gaines, 1977). Recent progress in these areas will be apparent in the sessions on systems modelling and complexity.…”

An initial categorization of foundational research in complex technical systems

“…GSS is concerned with generic problems (such as boundary, definition, coherence, complexity, and invariance) from both theoretical and practical perspectives (Jamshidi, 1996), and investigates generic (paradigmatic) system attributes (Gaines, 1977). It has explicated a system concept that seems to unify the interests of researchers in this area (Marchal, 1975).…”

Precise past—fuzzy future

“…This problem was solved for deterministic automata by Nerode (1958) and given a category theoretic formulation for general deterministic systems by Goguen (1973), Arbib & Manes (1974) and Ehrig (1974). It was solved by Gaines (1975bGaines ( , 1977 for stochastic automata and given a category theoretic formulation by Ralescu (1979) for general non-deterministic systems.…”

“…Gaines & Kohout (1975) and Zadeh (1978) have proposed that Lukasiewicz or fuzzy logic be regarded as a basis for a possibilistic system theory and given illustrations of its applications to database systems (Gaines, 1981a) and to the extension of probabilistic sequential system identification (Gaines, 1977) to uncertain sequential data (Gaines, 1979). The modal logic $5 is the classical logic of possibility of Lewis & Langford (1932).…”