Interaction Between Evolution and Learning in NK Fitness Landscapes

Graham, Todd; Candadai, Madhavun; Izquierdo, Eduardo J.

doi:10.1162/isal_a_00331

Cited by 5 publications

(4 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The main motivation for combining evolution and deep RL is the improved performance that may result from the combination. For instance, through simple experiments with simple fitness landscapes and simplified versions of the components, combining evolution and RL can be shown to work better than using either of the two in isolation (Todd et al, 2020). Why is this so?…”

Section: Evolution Of Policies For Performancementioning

confidence: 99%

Combining Evolution and Deep Reinforcement Learning for Policy Search: a Survey

Sigaud¹

2022

Preprint

View full text Add to dashboard Cite

Deep neuroevolution and deep Reinforcement Learning have received a lot of attention in the last years. Some works have compared them, highlighting theirs pros and cons, but an emerging trend consists in combining them so as to benefit from the best of both worlds. In this paper, we provide a survey of this emerging trend by organizing the literature into related groups of works and casting all the existing combinations in each group into a generic framework. We systematically cover all easily available papers irrespective of their publication status, focusing on the combination mechanisms rather than on the experimental results. In total, we cover 43 algorithms more recent than 2017. We hope this effort will favor the growth of the domain by facilitating the understanding of the relationships between the methods, favoring deeper analyses, outlining missing useful comparisons and suggesting new combinations of mechanisms.

show abstract

Section: Evolution Of Policies For Performancementioning

confidence: 99%

Combining Evolution and Deep Reinforcement Learning for Policy Search: a Survey

Sigaud¹

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Increasingly, work combining evolution and lifetime learning mechanisms have been proposed and explored (Todd et al, 2020;Gupta et al, 2021) and this offers yet another opportunity to combine lifetime learning and evolution. The synergy of these two forms of learning might enable training dynamical systems previously too difficult to train using purely lifetime learning or evolutionary approaches alone.…”

Section: Combining Lifetime and Evolutionary Learningmentioning

confidence: 99%

Reinforcement Learning for Central Pattern Generation in Dynamical Recurrent Neural Networks

Yoder

Anderson

Wang

et al. 2022

Front. Comput. Neurosci.

View full text Add to dashboard Cite

Lifetime learning, or the change (or acquisition) of behaviors during a lifetime, based on experience, is a hallmark of living organisms. Multiple mechanisms may be involved, but biological neural circuits have repeatedly demonstrated a vital role in the learning process. These neural circuits are recurrent, dynamic, and non-linear and models of neural circuits employed in neuroscience and neuroethology tend to involve, accordingly, continuous-time, non-linear, and recurrently interconnected components. Currently, the main approach for finding configurations of dynamical recurrent neural networks that demonstrate behaviors of interest is using stochastic search techniques, such as evolutionary algorithms. In an evolutionary algorithm, these dynamic recurrent neural networks are evolved to perform the behavior over multiple generations, through selection, inheritance, and mutation, across a population of solutions. Although, these systems can be evolved to exhibit lifetime learning behavior, there are no explicit rules built into these dynamic recurrent neural networks that facilitate learning during their lifetime (e.g., reward signals). In this work, we examine a biologically plausible lifetime learning mechanism for dynamical recurrent neural networks. We focus on a recently proposed reinforcement learning mechanism inspired by neuromodulatory reward signals and ongoing fluctuations in synaptic strengths. Specifically, we extend one of the best-studied and most-commonly used dynamic recurrent neural networks to incorporate the reinforcement learning mechanism. First, we demonstrate that this extended dynamical system (model and learning mechanism) can autonomously learn to perform a central pattern generation task. Second, we compare the robustness and efficiency of the reinforcement learning rules in relation to two baseline models, a random walk and a hill-climbing walk through parameter space. Third, we systematically study the effect of the different meta-parameters of the learning mechanism on the behavioral learning performance. Finally, we report on preliminary results exploring the generality and scalability of this learning mechanism for dynamical neural networks as well as directions for future work.

show abstract

“…Particularly, learning from other members in a group generally leads to a collective success by exploiting verified information [1][2][3], which distinguishes social learning from asocial, individual learning where information directly comes from exploration through the environment. While social learning is intuitively beneficial at first sight, research over the past several decades has consistently proven that naive social imitation is not inherently adaptive and often fails to achieve good group-level performance [4][5][6][7][8][9][10]. Instead, current theory suggests that, to properly determine how to learn from others one should employ a selective heuristics called a social learning strategy (SLS) [5], which governs the internal rules for choosing the proper time, subject, and methods to engage both social and individual learning.…”

Section: Introductionmentioning

confidence: 99%

“…In particular, RL has shown its strength in constructing computational models of the spatiotemporal dynamics of human cooperation and anthropological problems [32][33][34][35]. Here, we model the problem of social learning by considering a group of individuals that iteratively search for better solutions in a rugged, high-dimensional landscape [6,9,10,18,19,36], where our goal is to find the optimal heuristic for individuals that yields the maximum average payoff when shared with its group (Fig. 1A).…”

Section: Introductionmentioning

confidence: 99%

Social learning spontaneously emerges by searching optimal heuristics with deep reinforcement learning

Ha¹,

Jeong²

2022

Preprint

View full text Add to dashboard Cite

How have individuals of social animals in nature evolved to learn from each other, and what would be the optimal strategy for such learning in a specific environment? Here, we address both problems by employing a deep reinforcement learning model to optimize the social learning strategies (SLSs) of agents in a cooperative game in a multi-dimensional landscape. Throughout the training for maximizing the overall payoff, we find that the agent spontaneously learns various concepts of social learning, such as copying, focusing on frequent and well-performing neighbors, self-comparison, and the importance of balancing between individual and social learning, without any explicit guidance or prior knowledge about the system. The SLS from a fully trained agent outperforms all of the traditional, baseline SLSs in terms of mean payoff. We demonstrate the superior performance of the reinforcement learning agent in various environments, including temporally changing environments and real social networks, which also verifies the adaptability of our framework to different social settings.

show abstract

Interaction Between Evolution and Learning in NK Fitness Landscapes

Cited by 5 publications

References 0 publications

Combining Evolution and Deep Reinforcement Learning for Policy Search: a Survey

Combining Evolution and Deep Reinforcement Learning for Policy Search: a Survey

Reinforcement Learning for Central Pattern Generation in Dynamical Recurrent Neural Networks

Social learning spontaneously emerges by searching optimal heuristics with deep reinforcement learning

Contact Info

Product

Resources

About