Effective Acceptance Strategy Using Deep Reinforcement Learning in Bilateral Multi-issue Negotiation

Abstract

Recently, automated negotiation has been attracting attention in multi-agent systems to resolve conflicts and reach an agreement among agents. In automated negotiation, two main types of strategies are incorporated in each agent: a bidding strategy that considers what kind of bid to send to an opponent, and an acceptance strategy that considers whether to accept the opponent's offer. In most bilateral multi-issue negotiation, agents take turns sending bids to each other and the negotiation ends when an agent accepts an opponent's offer. Therefore, the acceptance strategy is important in terms of increasing the utility of an agent. However, most studies of automated negotiation using reinforcement learning focus only on the bidding strategy of the agent, so there are not many studies that investigate acceptance strategies using reinforcement learning. In this paper, we propose a new configuration of a deep reinforcement learning framework for the acceptance strategy in automated negotiations using Deep Q-Network. The training phase is performed multiple times with various reward functions, and the reward capable of a higher utility value is investigated. Simulation experiments with other negotiating agents showed that the proposed method obtained significantly higher utility values than existing methods.