A further generalization of the finite-population geiringer-like theorem for pomdps to allow recombination overarbitrary set covers

Abstract

Funder: EPSRC RONO: EP/I009809/1A popular current research trend deals with expanding the Monte-Carlo tree search sampling methodologies to the environments with uncertainty and incomplete information. Recently a finite population version of Geiringer theorem with nonhomologous recombination has been adopted to the setting of Monte-Carlo tree search to cope with randomness and incomplete information by exploiting the entrinsic similarities within the state space of the problem. The only limitation of the new theorem is that the similarity relation was assumed to be an equivalence relation on the set of states. In the current paper we lift this "curtain of limitation" by allowing the similarity relation to be modeled in terms of an arbitrary set cover of the set of state-action pairs.Peer reviewe

Geiringer theorems: from population genetics to computational intelligence, memory evolutive systems and Hebbian learning

“…It is conceivable then that the action evaluation mechanisms for the collective links towards the motor-related clusters of neurons are implemented in a similar manner as described at the end of the previous section. Moreover, the newest generalized Geiringer-like theorem in [14] that allows recombination over arbitrary set cover, leads to new fascinating insights into payoff-based clustering algorithms that resemble self-organizing maps and may also play a significant role in the conceptformation. These algorithms are the subject of sequel papers.…”

“…Not only such algorithms are anticipated to have a tremendous impact when coping with POMDPs (partially observable Markov decision processes), a link has been exhibited between such algorithms for decision making in the environments with randomness and incomplete information and the corresponding algorithms in biological neural networks through an elegant and well-developed category-theoretic model of cognition in [18]. The most recent version of the theorem in [14] motivates further extensions of the dynamic algorithms in the current paper for payoff-based clustering and will be investigated in the sequel papers.…”

“…The technical challenges have been overcome thanks to a lovely application of the classical and extremely useful triviality known as the Markov inequality (see, for instance, [8]) and enhancing the methodology for deriving the Geiringer-like results in [9] through exploiting a slightly extended lumping quotients of Markov chains technique that has been successfully developed, improved and exploited to estimate stationary distributions of Markov chains modeling EAs in a series of articles: [10], [11], [12] and [13]. Finally, the later version of the theorem has been further generalized to allow recombination over arbitrary set covers, rather than being limited to equivalence relations using the particular case in [7] in conjunction with the tools mentioned above in [14]. While the original purpose of the last two finite-population Geiriger-like theorems is taking advantage of the intrinsic similarities within the state-action set encountered by a learning agent to evaluate actions with the aim of selecting an optimal one, the parallel algorithms on the evolving digraph where the nodes are states and actions are edges from a current state to the one obtained upon executing an action, that are motivated by the theorems exhibit similarities to the way Hebbian learning takes place in biological neural networks that is further supported by Andree Ehresmann's category-theoretic model of cognitive processes called a "Memory Evolutive System" (see, for instance, [15], [16], [17], [18] and many more related articles).…”