A game theoretic framework for incentive-based models of intrinsic motivation in artificial systems

Abstract

An emerging body of research is focusing on understanding and building artificial systems that can achieve open-ended development influenced by intrinsic motivations. In particular, research in robotics and machine learning is yielding systems and algorithms with increasing capacity for self-directed learning and autonomy. Traditional software architectures and algorithms are being augmented with intrinsic motivations to drive cumulative acquisition of knowledge and skills. Intrinsic motivations have recently been considered in reinforcement learning, active learning and supervised learning settings among others. This paper considers game theory as a novel setting for intrinsic motivation. A game theoretic framework for intrinsic motivation is formulated by introducing the concept of optimally motivating incentive as a lens through which players perceive a game. Transformations of four well-known mixed-motive games are presented to demonstrate the perceived games when players' optimally motivating incentive falls in three cases corresponding to strong power, affiliation and achievement motivation. We use agent-based simulations to demonstrate that players with different optimally motivating incentive act differently as a result of their altered perception of the game. We discuss the implications of these results both for modeling human behavior and for designing artificial agents or robots.

Computational Motivation, Autonomy and Trustworthiness: Can We Have It All?

“…The remainder of the algorithm proceeds as follows: Each agent in the swarm is initialized with an OMI, Ω j (line 1) [30]. At each time step, each agent senses the local state of its environment (line 4), including the features described above for position, velocity and neighbors within different ranges.…”

A Survey of Game Theoretic Approaches to Modelling Decision-Making in Information Warfare Scenarios

“…However, a number of studies have shown that decision makers in many real situations do not adhere to these assumptions [90]. This has led to an increasing recognition that it is necessary to model players in greater detail, considering their behavioural characteristics including intent, objectives, strategies [5] and motives [6]. A few different branches have emerged in game theory literature that specifically focus on bounding the rationality assumption.…”

“…These games are analysed to establish their equilibrium points and suggest beneficial strategies for players. However, it is increasingly recognised that it is necessary to model players in greater detail, including their intent, objectives and strategies [5] and their motives [6]. This paper presents a taxonomy of games that map to information warfare and cyber crime problems, as a precursor to future research on decision-making in such scenarios.…”

The Role of Implicit Motives in Strategic Decision-Making: Computational Models of Motivated Learning and the Evolution of Motivated Agents

“…In humans, differences in implicit motives have also been linked to differences in preferences for explicit incentives such as money, points or "payoff" in a game [2][3][4][5] or during other kinds of strategic interactions [6]. Computational motivation has emerged as an area of interest among artificial intelligence [7,8] and robotics [9][10][11] researchers, as a mechanism for enabling autonomous mental development [12] or modelling human-like decision-making [8,13]. This paper falls in this latter category.…”

“…Later work simplifies this kind of model for game-theoretic settings using the concept of an optimally motivating incentive (OMI) [8,23]. The OMI is the incentive value that maximizes a motivation curve.…”