Playing Against Fair Adversaries in Stochastic Games with Total Rewards

Abstract

We investigate zero-sum turn-based two-player stochastic games in which the objective of one player is to maximize the amount of rewards obtained during a play, while the other aims at minimizing it. We focus on games in which the minimizer plays in a fair way. We believe that these kinds of games enjoy interesting applications in software verification, where the maximizer plays the role of a system intending to maximize the number of “milestones” achieved, and the minimizer represents the behavior of some uncooperative but yet fair environment. Normally, to study total reward properties, games are requested to be stopping (i.e., they reach a terminal state with probability 1). We relax the property to request that the game is stopping only under a fair minimizing player. We prove that these games are determined, i.e., each state of the game has a value defined. Furthermore, we show that both players have memoryless and deterministic optimal strategies, and the game value can be computed by approximating the greatest-fixed point of a set of functional equations. We implemented our approach in a prototype tool, and evaluated it on an illustrating example and an Unmanned Aerial Vehicle case study.

Quantifying Masking Fault-Tolerance via Fair Stochastic Games

“…The next lemma states that, if the Refuter plays a semi-Markov strategy, the Verifier achieves equal results regardless whether she plays an arbitrary strategy or limits to playing only semi-Markov strategies. The proof resembles that of [9,Lemma 2] taking care of the fact that the set of vertices of the stochastic masking game is uncountable. Since probabilities are anyway discrete, this is not a major technical issue, but it deserves attention in the proof.…”

“…Then, a fair strategy for the Refuter is a strategy that always measures 1 on the set of all the Refuter's fair plays, regardless of the strategy of the Verifier. The definitions below follow the style in [5,4,9].…”

“…Interestingly, under the strong fairness assumption, the determinacy of games is preserved for finite stochastic games [9]. The rest of the section is precisely devoted to bring our setting to the framework of [9] and thus provide an algorithmic solution.…”

“…Proof. We first recall that the almost-sure failing under fairness property is equivalent to the stopping under fairness property in [9]. That is why we can safely apply the results from [9] on H A,A ′ in the calculations below.…”

“…As our game is of infinite nature, the results in [9] cannot be applied directly. Therefore we devise a finite discretization that allows us to partly reuse [9] and show that the value of the game is determined Borel σ -algebra, Prob π 1 ,π 2 G ,v (A ) is the probability that strategies π 1 and π 2 generate a play belonging to A from state v. It would normally be convenient to use LTL notation to define events. For instance,…”