Co-evolution of language and agents in referential games

Abstract

Referential games offer a grounded learning environment for neural agents which accounts for the fact that language is functionally used to communicate. However, they do not take into account a second constraint considered to be fundamental for the shape of human language: that it must be learnable by new language learners. 2019) introduced cultural transmission within referential games through a changing population of agents to constrain the emerging language to be learnable. However, the resulting languages remain inherently biased by the agents' underlying capabilities. Cogswell et al. (In this work, we introduce Language Transmission Simulator to model both cultural and architectural evolution in a population of agents. As our core contribution, we empirically show that the optimal situation is to take into account also the learning biases of the language learners and thus let language and agents coevolve. When we allow the agent population to evolve through architectural evolution, we achieve across the board improvements on all considered metrics and surpass the gains made with cultural transmission. These results stress the importance of studying the underlying agent architecture and pave the way to investigate the co-evolution of language and agent in language emergence studies.

RLupus: Cooperation through emergent communication in The Werewolf social deduction game

“…Although there are several variations of the discrimination referential game, as illustrated in Table 1, the primary focus of research has been analyzing the language that emerges from it. This has been demonstrated in various studies, including [Yuan et al, 2020, Rodriguez et al, 2019, Graesser et al, 2019, Li and Bowling, 2019, Dagan et al, 2020, Havrylov and Titov, 2017, Lazaridou et al, 2017, Wang et al, 2021.…”

“…This requires the integration of multiple learning aspects within a single framework as well as exploring new learning paradigms. However, in the literature analyzed for this review, a surprisingly limited number of studies explore alternative learning approaches, such as unsupervised learning [Grover et al, 2018, Mao et al, 2019, evolution strategies [Dagan et al, 2020], stochastic computational graphs and self-supervised learning [Dessì et al, 2021].…”

“…Indeed, reinforcement learning is a commonly used training paradigm for iterative learning, with Self Play [Tesauro, 1994] being the preferred approach [Lowe et al, 2020, Graesser et al, 2019, Gupta et al, 2019. Other approaches include seeded iterative learning [Lu et al, 2020b,a], and evolution strategies [Dagan et al, 2020].…”

“…Similarly, the message distribution's entropy is reported in [Choi et al, 2018, Graesser et al, 2019, Chaabouni et al, 2019a, Dagan et al, 2020, Chaabouni et al, 2019b, Wang et al, 2019 as a measure of the correlation between the speaker's input and the message used to describe it. When the entropy is low, the speaker is consistently using the same message to describe that input, thus showing some kind of communication protocol.…”

Communicative Feedback in language acquisition

“…While such models have studied language creation/emergence, very similar computational tools can be used, in principle, to study language acquisition. In fact, some studies in this same literature successfully incorporated a language transmission component (from a pre-trained "teacher" to an untrained "student") in their multi-generational emergent communication frameworks (Cogswell et al, 2020;Dagan et al, 2021;Li & Bowling, 2019;Lu et al, 2020). That said, the goals of these studies has been still the study of language emergence across generations rather than the study of language acquisition of a child in an interactive context.…”