Computing the Stackelberg/Nash equilibria using the extraproximal method: Convergence analysis and implementation details for Markov chains games

Abstract

In this paper we present the extraproximal method for computing the Stackelberg/Nash equilibria in a class of ergodic controlled finite Markov chains games. We exemplify the original game formulation in terms of coupled nonlinear programming problems implementing the Lagrange principle. In addition, Tikhonov's regularization method is employed to ensure the convergence of the cost-functions to a Stackelberg/Nash equilibrium point. Then, we transform the problem into a system of equations in the proximal format. We present a two-step iterated procedure for solving the extraproximal method: (a) the first step (the extra-proximal step) consists of a "prediction" which calculates the preliminary position approximation to the equilibrium point, and (b) the second step is designed to find a "basic adjustment" of the previous prediction. The procedure is called the "extraproximal method" because of the use of an extrapolation. Each equation in this system is an optimization problem for which the necessary and efficient condition for a minimum is solved using a quadratic programming method. This solution approach provides a drastically quicker rate of convergence to the equilibrium point. We present the analysis of the convergence as well the rate of convergence of the method, which is one of the main results of this paper. Additionally, the extraproximal method is developed in terms of Markov chains for Stackelberg games. Our goal is to analyze completely a three-player Stackelberg game consisting of a leader and two followers. We provide all the details needed to implement the extraproximal method in an efficient and numerically stable way. For instance, a numerical technique is presented for computing the first step parameter (λ) of the extraproximal method. The usefulness of the approach is successfully demonstrated by a numerical example related to a pricing oligopoly model for airlines companies.

An optimal strong equilibrium solution for cooperative multi-leader-follower Stackelberg Markov chains games

“…The Extraproximal Method for the conditional optimization problems (25) was suggested in ( [2,30]). We design the method for the static Stackelberg-Nash game in a general format.…”

“…Then, we reduce the optimization problem to find a Pareto optimal solution. We employ a bi-level programming model implemented by the extraproximal optimization approach [2,30] for computing the strong Stackelberg/Nash equilibrium. The extraproximal approach is a natural extension of the proximal and the gradient optimization methods used for solving the more difficult problems for finding an equilibrium point in game theory.…”

“…See [30] Theorem 5.2. (Convergence and Uniqueness) LetL δ (ũ,z) be differentiable inũ andz, whose partial derivative with respect toz satisfies the Lipschitz condition with positive constant C. Then, for some δ and…”

“…The dynamics of a Stackelberg game is as follows ( [30]): The leader considers the best-reply of the follower. Then, he/she commits to a mixed strategy (a probability distribution over deterministic schedules) that minimizes the cost, anticipating the predicted best-reply of the follower.…”

Negotiating transfer pricing using the Nash bargaining solution

“…We start with some notation (Clempner and Poznyak, 2011;2016;Trejo et al, 2015). For a finite set of divisions (players) N with n elements, let R n denote the n-dimensional Euclidean space with coordinates indexed by l = 1, .…”