We develop a theoretical framework for the study of cvolutionary learning systems. The formalism we use is that of history dependent stochastic automata with suitable structure, as well as related structures. This formalism provides a natural setting in which to describe the learning of classification hierarchies, of control hierarchies and notions of selfrefercnce, all of which are derived as consequences of the ability to learn by association.