Intrinsically motivated collective motion

Abstract

Collective motion is found in various animal systems, active suspensions and robotic or virtual agents. This is often understood using high level models that directly encode selected empirical features, such as co-alignment and cohesion. Can these features be shown to emerge from an underlying, low-level principle? We find that they emerge naturally under Future State Maximisation (FSM). Here agents perceive a visual representation of the world around them, such as might be recorded on a simple retina, and then move to maximise the number of different visual environments that they expect to be able to access in the future. Such a control principle may confer evolutionary fitness in an uncertain world by enabling agents to deal with a wide variety of future scenarios. The collective dynamics that spontaneously emerge under FSM resemble animal systems in several qualitative aspects, including cohesion, co-alignment and collision suppression, none of which are explicitly encoded in the model. A multi-layered neural network trained on simulated trajectories is shown to represent a heuristic mimicking FSM. Similar levels of reasoning would seem to be accessible under animal cognition, demonstrating a possible route to the emergence of collective motion in social animals directly from the control principle underlying FSM. Such models may also be good candidates for encoding into possible future realisations of artificial "intelligent" matter, able to sense light, process information and move.

Machine learning for active matter

“…In dilute systems, when active particles have no direct physical interaction with each other, natural systems have evolved sensing capabilities, which allow them to gain information about their environments or to communicate. This introduces a new dimension with entirely different challenges and importance in other fields such as ecology 18,19 . In schools of fish and flocks of birds or midges, individuals exchange information as part of their behaviour to self-organize into a collective state 20 .…”

Efficient Navigation of Colloidal Robots in an Unknown Environment via Deep Reinforcement Learning

“…Our DRL architecture can also be used as a basic building block to construct more complex learning architectures for navigation tasks in more challenging environments, including incorporating additional visual channels for navigation in flow fields, adding memory module for navigation in nonstationary environments with limited visibility, using 3D convolutional layers for 3D navigation, extending to continuous control DRL for high‐precision localization tasks, and building hierarchical neural networks for navigation in environments with multiple‐scale obstacle features. Our algorithm can also be extended to a multiagent system to control multiple robots to cooperate on tasks and assemble to nonequilibrium machines and devices or applied as a general end‐to‐end controller for controlling stochastic colloidal assembly . Ultimately, our DRL algorithm allows to be integrated with experimental systems as it can directly process raw sensor inputs of microrobot systems (e.g., microscope) .…”

Goal-Directed Empowerment: Combining Intrinsic Motivation and Task-Oriented Behavior

“…This concept of preparedness motivates the introduction of empowerment as information-theoretic quantification and formalization of the amount of open options that an agent can both control and perceive. The principle of empowerment maximisation has been applied to very different areas of artificial intelligence [1], [6], [7], [30], [31], robotics and control [1], [4], [5], [13]. Importantly, empowerment was defined in the same fashion across all these experiments.…”