Entropic fictitious play for mean field optimization problem

Abstract

It is well known that the training of the neural network can be viewed as a mean field optimization problem. In this paper we are inspired by the fictitious play, a classical algorithm in the game theory for learning the Nash equilibria, and propose a new algorithm, different from the conventional gradient-descent ones, to solve the mean field optimization. We rigorously prove its (exponential) convergence, and show some simple numerical examples.

Primal and Dual Analysis of Entropic Fictitious Play for Finite-sum Problems

“…Remark 1. Under Assumption 1, the existence and uniqueness of the minimizer µ * ∈ P 2 of L is guaranteed by Ren and Wang (2022), and µ * satisfies the first-order optimality condition: µ * = μ * (Hu et al, 2019).…”

“…The entropic fictitious play (EFP) algorithm (Ren and Wang, 2022) is the optimization method that minimizes the objective (4). The continuous evolution {µ t } t≥0 ⊂ P 2 of the EFP is defined as follows: for γ > 0,…”

“…In this work, we analyze a different update rule that optimizes (1) recently proposed in Ren and Wang (2022) termed the entropic fictitious play (EFP). The EFP update is inspired by the classical fictitious play in Figure 1: 1D visualization of the parameter distribution of mean-field two-layer neural network (tanh) optimized by the mean-field Langevin dynamics (MFLD), particle dual averaging 1 (PDA), and entropic fictitious play (EFP), all of which can solve the self-consistent equation µ = μ, where μ is the proximal Gibbs measure associated with µ.…”

Polyak-Łojasiewicz inequality on the space of measures and convergence of mean-field birth-death processes

“…Such optimization problems have attracted considerable attention in recent years, see e.g. [9,16,12,18,7]. In this setting, there exist multiple different choices of flows of probability measures (m t ) t≥0 that can serve as analogues of the gradient descent algorithm in R d , as well as multiple different choices of conditions on V analogous to (1.1) that can be used to prove convergence of such flows.…”