Reinforcement learning and artificial agency

Abstract

There is an apparent connection between reinforcement learning and agency. Artificial entities controlled by reinforcement learning algorithms are standardly referred to as agents, and the mainstream view in the psychology and neuroscience of agency is that humans and other animals are reinforcement learners. This article examines this connection, focusing on artificial reinforcement learning systems and assuming that there are various forms of agency. Artificial reinforcement learning systems satisfy plausible conditions for minimal agency, and those which use models of the environment to perform forward search are capable of a form of agency which may reasonably be called action for reasons.

Understanding Artificial Agency

“…One is an intentional agent in this sense if and only if one is able to act for reasons. Butlin (2023) argues that model-based reinforcement learning (RL) systems count as acting for reasons. The state transition function in a RL training environment specifies which states follow (with a particular probability) from which actions, given the states in which the actions have been produced.…”

“…Model-based RL systems represent the state transition function along with the value of different states and use this information to choose which action to output. Consequently, Butlin claims, 'they represent and act on the basis of facts which count in favour of their actions, given their goals, and therefore […] act for reasons' (Butlin 2023).…”