Deep reinforcement learning for acceptance strategy in bilateral negotiations

Abstract

This paper introduces an acceptance strategy based on reinforcement learning for automated bilateral negotiation, where negotiating agents bargain on multiple issues in a variety of negotiation scenarios. Several acceptance strategies based on predefined rules have been introduced in the automated negotiation literature. Those rules mostly rely on some heuristics, which take time and/or utility into account. For some negotiation settings, an acceptance strategy solely based on a negotiation deadline might perform well; however, it might fail in another setting. Instead of following predefined acceptance rules, this paper presents an acceptance strategy that aims to learn whether to accept its opponent's offer or make a counter offer by reinforcement signals received after performing an action. In an experimental setup, it is shown that the performance of the proposed approach improves over time.

Deep Learnable Strategy Templates for Multi-Issue Bilateral Negotiation

“…Existing approaches with reinforcement learning have focused on methods such as Tabular Q-learning for bidding [12] and finding the optimal concession [34,35] or DQN for bid acceptance [27], which are not optimal for continuous action spaces. Such spaces, however, are the main focus in this work in order to estimate the threshold target utility value below which no bid is accepted/proposed from/to the opponent agent.…”

“…(b) Meta-heuristic (or evolutionary) methods -work well across domains and improve iteratively using a fitness function (as a guide for quality); however, in these approaches every time an agent decision is made, this needs to be delivered by the meta-heuristic, which is not efficient and does not result in a human-interpretable and reusable negotiation strategy. (c) Machine learning algorithms -they show the best results with respect to run-time adaptability [8,27], but often their working hypotheses are not interpretable, a fact that may hinder their eventual adoption by users due to lack of transparency in the decision-making that they offer. (d) Interpretable strategy templates -developed in [10] to guide the use of a series of tactics whose optimal use can be learned during negotiation.…”

ANEGMA: an automated negotiation model for e-markets

“…This is not, however, an ideal solution for large state/action spaces as it may lead to the curse of dimensionality, as well as cause the loss of relevant information about the state/action domain structure. Razeghi et al [43] use Deep Q Networks [36] to design a learnable acceptance strategy based on feedback received from the environment. The main difference of the above Q-learning approaches, when compared to ours, is that they cannot be used in continuous action spaces [31], and thus are inappropriate for our setting.…”

An Autonomous Negotiating Agent Framework with Reinforcement Learning Based Strategies and Adaptive Strategy Switching Mechanism

“…Then again, in the last couple of decades several studies have looked at the application of reinforcement learning (RL) algorithms like Q-learning [17,20,46,48,49] and REINFORCE [47] in automated negotiation. Recently, Deep Reinforcement learning (DRL) has been used to learn the target utility values [16], the acceptance strategy [43] or both bidding and acceptance strategies [19]. Moreover, authors of [15] have also shown application of DRL in concurrent bilateral negotiation.…”