Controlled Markov Processes With Safety State Constraints

Tolling for Constraint Satisfaction in Markov Decision Process Congestion Games

“…Hence we consider a scenario where the game designer must deduce the optimal toll value by observing and interacting with a group of decision makers. By incorporating results from constrained traffic assignment literature [7,8] with MDP algorithms [9,10,11], we formulate arXiv:1907.08912v1 [cs.GT] 21 Jul 2019 a myopic toll update based on the population distribution of decision makers converging to an MDP Wardrop equilibrium. Under the interactive scenario between game designer and decision makers, we show that the toll update converges to the optimal tolling values.…”

“…where δ 2 ≤ , and¯ defined by (11), then y(J) satisfies -equilibrium at penalty p, denoted by y(J) ∈ W ρ (p, ). For any y ∈ W ρ (p), Definition 2 is trivially satisfied.…”

“…s j is a neighbouring state of s i if a direct route from s i to s j exists without passing through other states. The following states are characterized as residential: 'Ballard' (3), 'Fremont' (4), 'Sand Point' (8), 'Magnolia' (9), 'Green Lake' (11), 'Ravenna' (12). Assume drivers have equal probabilities of starting from any of the residential neighbourhoods.…”

“…In particular, the update for w k+1 is the analytical solution to argmin w∈R I + L ρ (y, w k , τ k ). The update for y k+1 is equivalent to finding an MDP Wardrop equilibrium for a game played with state-action costs given by (11) when σ(i) = τ k (i) + ρw k (i).…”

“…After each toll adjustment, the designer announces a toll update, and the decision makers converge to an MDP Wardrop equilibrium based on the new toll. Algorithm 5 shows that such a toll adjustment algorithm will result in convergence of τ k to the optimal tolling value τ , convergence of y k to the MDP Wardrop equilbrium of game with state-action costs given by (11).…”