How a minimal learning agent can infer the existence of unobserved variables in a complex environment

Ried, Katja; Eva, Benjamin; Müller, Thomas; Briegel, Hans J.

doi:10.48550/arxiv.1910.06985

Cited by 4 publications

(11 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A wide range of models and techniques have been applied to the study of collective behavior. In this work, we apply Projective Simulation, a model for artificial agency [9,[31][32][33][34][35]. Each individual is an artificial agent that can perceive its surroundings, make decisions and perform actions.…”

Section: The Model and The Learning Setupmentioning

confidence: 99%

“…In the latter case, the agents leave the swarm continuously, so the average number of neighbors decreases slowly until the swarm is completely dissolved. For d F = 2 (d F = 4) the individual responses are such that the average number of neighbors increases (decreases) in the first 30 rounds until the swarm stabilizes and from then on M stays at a stable value of 57 (35) neighbors. The average number of neighbors is correlated to the swarm size, which we measure by the difference between the maximum and minimum world positions occupied by the agents (modulo world 10 The agents do not learn anything new in these simulations, i.e.…”

Section: Cohesionmentioning

confidence: 99%

See 1 more Smart Citation

Development of swarm behavior in artificial learning agents that adapt to different foraging environments

et al. 2020

Self Cite

View full text Add to dashboard Cite

Collective behavior, and swarm formation in particular, has been studied from several perspectives within a large variety of fields, ranging from biology to physics. In this work, we apply Projective Simulation to model each individual as an artificial learning agent that interacts with its neighbors and surroundings in order to make decisions and learn from them. Within a reinforcement learning framework, we discuss one-dimensional learning scenarios where agents need to get to food resources to be rewarded. We observe how different types of collective motion emerge depending on the distance the agents need to travel to reach the resources. For instance, strongly aligned swarms emerge when the food source is placed far away from the region where agents are situated initially. In addition, we study the properties of the individual trajectories that occur within the different types of emergent collective dynamics. Agents trained to find distant resources exhibit individual trajectories that are in most cases best fit by composite correlated random walks with features that resemble Lévy walks. This composite motion emerges from the collective behavior developed under the specific foraging selection pressures. On the other hand, agents trained to reach nearby resources predominantly exhibit Brownian trajectories.

show abstract

Section: The Model and The Learning Setupmentioning

confidence: 99%

Section: Cohesionmentioning

confidence: 99%

Development of swarm behavior in artificial learning agents that adapt to different foraging environments

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…Projective Simulation (PS) [8,[16][17][18][19][20][21] is a model for artificial agency that combines a notion of episodic memory with a simple reinforcement learning mechanism. It allows an agent to adapt its internal decision making processes and improve its performance in a given environment.…”

Section: B Theoretical Model: Projective Simulationmentioning

confidence: 99%

“…The ECM has a flexible structure that may consist of several layers and that can change over time by, for instance, the creation of new clips and their addition to the existing network (see e.g. [18,20]). However, for the purpose of this work, it is sufficient to consider the basic two-layered structure (see Fig.…”

Section: B Theoretical Model: Projective Simulationmentioning

confidence: 99%

Collective defense of honeybee colonies: experimental results and theoretical modeling

López-Incera,

Nouvian,

Ried

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Social insect colonies routinely face large vertebrate predators, against which they need to mount a collective defense. To do so, honeybees use an alarm pheromone that recruits nearby bees into mass stinging of the perceived threat. This alarm pheromone is carried directly on the stinger, hence its concentration builds up during the course of the attack. Here, we investigate how individual bees react to different alarm pheromone concentrations, and how this evolved response-pattern leads to better coordination at the group level. We first present an individual dose-response curve to the alarm pheromone, obtained experimentally. Second, we apply Projective Simulation to model each bee as an artificial learning agent that relies on the pheromone concentration to decide whether to sting or not. If the emergent collective performance benefits the colony, the individual reactions that led to it are enhanced via reinforcement learning, thus emulating natural selection. Predators are modeled in a realistic way so that the effect of factors such as their resistance, their killing rate or their frequency of attacks can be studied. We are able to reproduce the experimentally measured response-pattern of real bees, and to identify the main selection pressures that shaped it. Finally, we apply the model to a case study: by tuning the parameters to represent the environmental conditions of European or African bees, we can predict the difference in aggressiveness observed between these two subspecies.

show abstract

“…Recently, the potential role that machine learning might play in the scientific discovery process has received increasing attention [15][16][17][18][19][20][21][22]. This direction of research is not only concerned with machine learning as a useful numerical tool for solving hard problems, but also seeks ways to establish artificial intelligence methodologies as general-purpose tools for scientific research.…”

Section: Introductionmentioning

confidence: 99%

Operationally meaningful representations of physical systems in neural networks

Nautrup

Metger

Iten

et al. 2022

Mach. Learn.: Sci. Technol.

View full text Add to dashboard Cite

To make progress in science, we often build abstract representations of physical systems that meaningfully encode information about the systems. Such representations ignore redundant features and treat parameters such as velocity and position separately because they can be useful for making statements about diﬀerent experimental settings. Here, we capture this notion by formally deﬁning the concept of operationally meaningful representations. We present an autoencoder architecture with attention mechanism that can generate such representations and demonstrate it on examples involving both classical and quantum physics. For instance, our architecture ﬁnds a compact representation of an arbitrary two-qubit system that separates local parameters from parameters describing quantum correlations.

show abstract

How a minimal learning agent can infer the existence of unobserved variables in a complex environment

Cited by 4 publications

References 27 publications

Development of swarm behavior in artificial learning agents that adapt to different foraging environments

Development of swarm behavior in artificial learning agents that adapt to different foraging environments

Collective defense of honeybee colonies: experimental results and theoretical modeling

Operationally meaningful representations of physical systems in neural networks

Contact Info

Product

Resources

About