Polynomial Time Algorithms to Find an Approximate Competitive Equilibrium for Chores

Abstract

Competitive equilibrium with equal income (CEEI) is considered one of the best mechanisms to allocate a set of items among agents fairly and efficiently. In this paper, we study the computation of CEEI when items are chores that are disliked (negatively valued) by agents, under 1-homogeneous and concave utility functions which includes linear functions as a subcase. It is well-known that, even with linear utilities, the set of CEEI may be non-convex and disconnected, and the problem is PPAD-hard in the more general exchange model. In contrast to these negative results, we design FPTAS: A polynomial-time algorithm to compute ε-approximate CEEI where the running-time depends polynomially on 1 ε . Our algorithm relies on the recent characterization due to Bogomolnaia et al. (2017) of the CEEI set as exactly the KKT points of a non-convex minimization problem that have all coordinates non-zero. Due to this non-zero constraint, naïve gradient-based methods fail to find the desired local minima as they are attracted towards zero. We develop an exterior-point method that alternates between guessing non-zero KKT points and maximizing the objective along supporting hyperplanes at these points. We show that this procedure must converge quickly to an approximate KKT point which then can be mapped to an approximate CEEI; this exterior point method may be of independent interest.When utility functions are linear, we give explicit procedures for finding the exact iterates, and as a result show that a stronger form of approximate CEEI can be found in polynomial time. Finally, we note that our algorithm extends to the setting of un-equal incomes (CE), and to mixed manna with linear utilities where each agent may like (positively value) some items and dislike (negatively value) others.

Finite-Sample Analysis of Decentralized Q-Learning for Stochastic Games

“…where ν i is the random policy that assigns the uniform distribution on A i to each s. In words, agent i plays the baseline policy with probability 1 − ρ i , and plays all actions uniformly with probability ρ i /|A i |. We denote by Π the set of joint policies in the form of (10) for each agent, i.e., Π :…”

“…Proof of Lemma 2. Note that in the kth exploration phase, agents adopt the joint policy πk as defined in (10). Also by Assumptions 1 and 2, the (finite) Markov chain induced by the joint policy is irreducible and aperiodic.…”

“…Our next goal is to bound the approximation error of policy perturbation. Recall the definition of the randomized policy in (10), and consider the joint policies of all agents except i. With probability j =i (1 − ρ j ), all agents j = i end up playing their baseline policies, which results in…”

“…The same problem as in the tabular setting arises here: if all agents use (56) and select their actions with the -greedy method, the environment becomes nonstationary and the convergence of the θ i 's is not guaranteed. In the same spirit of Algorithm 1, we let each agent play the behavior policy πi k as defined in (10) during the kth exploration phase, so that the environment is stationary within each exploration phase. Instead of maintaining and updating a |S||A i |-dimensional Q function, agent i updates a d-dimensional vector θ i according to (56).…”

“…Roughly speaking, in a Markov perfect equilibrium, each agent's policy is a best reply (i.e., maximizes her own total (discounted) reward) to all other agents' joint policy. Mainstreams of research include analyzing the hardness to compute the equilibria (e.g., Daskalakis [17], Daskalakis et al [19], Garg et al [33]), approximating and analyzing the equilibria (e.g., Adsul et al [1], Boodaghians et al [10], Brânzei et al [11]), designing algorithms to find the equilibria with the knowledge of the transitions and rewards (e.g., Hansen et al [35], Hu and Wellman [39]) or without such knowledge (e.g. Arslan and Yüksel [3]).…”