Improving the Survival Time of Multiagents in Social Dilemmas through Neurotransmitter-Based Deep Q-Learning Model of Emotions

Abstract

In multiagent systems, social dilemmas often arise whenever there is a competition over the limited resources. The major challenge is to establish cooperation among intelligent virtual agents for solving the situations of social dilemmas. In humans, personality and emotions are the primary factors that lead them toward a cooperative environment. To make agents cooperate, they have to become more like humans, that is, believable. Therefore, we hypothesize that emotions according to the personality give birth to believability, and if believability is introduced into agents through emotions, it improves their survival rate in social dilemma situations. The existing researches have introduced different computational models to introduce emotions in virtual agents, but they lack emotions through neurotransmitters. We have proposed a neurotransmitters-based deep Q-learning computational model in multiagents that is a suitable choice for emotion modeling and, hence, believability. The proposed model regulates the agents’ emotions by controlling the virtual neurotransmitters (dopamine and oxytocin) according to the agent’s personality. The personality of the agent is introduced using OCEAN model. To evaluate the proposed system, we simulated a survival scenario with limited food resources in different experiments. These experiments vary the number of selfish agents (higher neuroticism personality trait) and the selfless agents (higher agreeableness personality trait). Experimental results show that by adding the selfless agents in the scenario, the agents develop cooperation, and their collective survival time increases. Thus, to resolve the social dilemma problems in virtual agents, we can make agents believable through the proposed neurotransmitter-based emotional model. This proposed work may help in developing nonplayer characters (NPCs) in games.

Cooperation is encouraged simply by increasing the number of participants

“…The last decade has seen a method termed reinforcement learning, which comprises learning the optimal strategy for a situation from experience, attract a great deal of attention. Indeed, Deep Q-Learning (DQL) [9][10][11] is in active use, and is an extension of the Q-Learning method involving the learning of the expected gain (Q-value) in the future when 'a certain action' is taken in 'a certain situation' using Deep Neural Network (DNN) 10 . The idea of the Q-value is similar to that of the sum of discounted expected gains in conventional iterative games 12 .…”