“If You Can't Be With the One You Love, Love the One You're With”: How Individual Habituation of Agent Interactions Improves Global Utility

Abstract

Simple distributed strategies that modify the behaviour of selfish individuals in a manner that enhances cooperation or global efficiency have proved difficult to identify. We consider a network of selfish agents who each optimise their individual utilities by coordinating (or anti-coordinating) with their neighbours, to maximise the pay-offs from randomly weighted pair-wise games. In general, agents will opt for the behaviour that is the best compromise (for them) of the many conflicting constraints created by their neighbours, but the attractors of the system as a whole will not maximise total utility. We then consider agents that act as 'creatures of habit' by increasing their preference to coordinate (anti-coordinate) with whichever neighbours they are coordinated (anti-coordinated) with at the present moment. These preferences change slowly while the system is repeatedly perturbed such that it settles to many different local attractors. We find that under these conditions, with each perturbation there is a progressively higher chance of the system settling to a configuration with high total utility. Eventually, only one attractor remains, and that attractor is very likely to maximise (or almost maximise) global utility. This counterintutitve result can be understood using theory from computational neuroscience; we show that this simple form of habituation is equivalent to Hebbian learning, and the improved optimisation of global utility that is observed results from wellknown generalisation capabilities of associative memory acting at the network scale. This causes the system of selfish agents, each acting individually but habitually, to collectively identify configurations that maximise total utility. Selfish Agents and Total UtilityThis paper investigates the effect of a simple distributed strategy for increasing total utility in systems of selfishly optimising individuals. The broader topic concerns many different types of systems. For example, in technological systems, it is often convenient or necessary to devolve control to numerous autonomous components or agents that each, in a fairly simple manner, acts to optimise a global performance criterion: e.g. communications routing agents act to minimise calls dropped, or processing nodes in a grid computing system each act to maximise the number of jobs processed (1,2). However, since each component in the network acts individually, i.e., using only local information, constraints between individuals can remain unsatisfied, resulting in poorly optimised global performance. In an engineered system one could, in principle, mandate that all nodes act in accord with the globally optimal configuration of behaviours (assuming one knew what that was) -but this would defeat the scalability and robustness aims of complex adaptive systems. The question for engineered complex adaptive systems then, is the question of how to cause simple autonomous agents to act 'smarter' in a fully distributed manner such that they better satisfy constraints between agents an...

Global Adaptation in Networks of Selfish Components: Emergent Associative Memory at the System Scale

“…In the social network domain we have also shown [7] that 'habituating' agents, acting as 'creatures of habit' due to a tendency to increase their preference for the 'status quo', are also formally equivalent to Hebbian learning at the system level and produce global adaptation in the same manner as that shown here. This work frames the utility function as a weighted poly-player coordination game [41,69,73], and the system dynamics as repeated episodes of attempting to reach consensus given constraints amongst players.…”

“…But when agents have the ability to alter who they play with, or the relative strength of their relationships with other agents, this is equivalent to changing the underlying game [72] (or changing how individuals perceive or experience the game [72,7]). Our question is not therefore, whether individual interests are aligned with global interests in the initial game we define, but rather whether the selfish restructuring of the interactions in the system leads, in effect, to agents that (because they are playing a modification of the underlying game) exhibit better alignment with total utility over time.…”

“…To ensure contractive dynamics, and to utilise formal treatments [78,81], we restrict our attention to symmetric connections, i.e. ω ij = ω ji [22] (this ensures that the total system is a potential game [7]). Note that the results presented above, i.e.…”

Evolutionary Connectionism: Algorithmic Principles Underlying the Evolution of Biological Organisation in Evo-Devo, Evo-Eco and Evolutionary Transitions

“…But in an ecological network this is not the case; reproduction of individuals simply modifies the strategy of the species at that node (and/or the connections the species has with others). We have shown that a games-on-networks model of this type behaves exactly like unsupervised correlation learning (Davies et al 2011; Watson et al 2010a). Moreover, since the total community welfare of a social attractor is correlated with the basin size of the attractor, the changes to connections produced under an individual utility-maximisation principle have the emergent effect of increasing global community welfare (collective fitness) without selection at the system level (Davies et al 2011; Watson et al 2010a)—as per a self-modelling dynamical system.…”

The web as an adaptive network

“…Other studies of games on networks allow individuals to adjust their social ties in a continuous-valued fullyconnected network [50,51]. For example, this might represent how user's perceptions of others change over time [50] or the probability of interaction [52] between two state affects how topology changes State (behavior) Topology (structure) topology affects how state changes players. These changes are equivalent to changing the (effective) strength of a connection or the weighting of a game between two players.…”