Stochastic Nestedness and the Belief Sharing Information Pattern

Abstract

Abstract-Solutions to decentralized stochastic optimization problems lead to recursions in which the state space enlarges with the time-horizon, thus leading to non-tractability of classical dynamic programming. A common joint information state supplied to each of the agents leads to a tractable recursion, as is evident in the one-step-delayed information sharing structure case or when deterministic nestedness in information holds when there is a causality relationship as in the case of partially nested information structure. However, communication requirements for such conditions require exchange of very large data noiselessly, hence these assumptions are generally impractical. In this paper, we present a weaker notion of nestedness, which we term as stochastic nestedness, which is characterized by a sequence of Markov chain conditions. It is shown that if the information structure is stochastically nested, then an optimization problem is tractable, and in particular for LQG problems, the team optimal solution is linear, despite the lack of deterministic nestedness or partial nestedness. One other contribution of this paper is that, by regarding the multiple decision makers as a single decision maker and using Witsenhausen's equivalent model for discrete-stochastic control, it is shown that the common state required need not consist of observations and it suffices to share beliefs on the state and control actions; a pattern we refer to as -stage belief sharing pattern. We discuss the minimum amount of information exchange required to achieve such an information pattern for = 1.The information exchange needed is generally strictly less than what is needed for deterministic nestedness and is zero whenever stochastic nestedness applies. In view of nestedness, we present a discussion on the monotone values of information channels.

Convex Analysis in Decentralized Stochastic Control, Strategic Measures, and Optimal Solutions

“…It is known that quasi-classical information structures eliminate the incentive for signaling, since the future decision makers already have access to the information at the signaling decision maker (see [43], but also [18], [50], [9] and [44] among other papers). On the other hand, one can also put a probabilistic flavor: [60] identified such a probabilistic, but rather restrictive, characterization; see also [36]. In the following, we exhibit that the static reduction provides an effective method to identify when lack of a signaling incentive can be established and can lead to a more refined probability and information structure dependent characterization of nestedness, that encompasses partial nestedness which is a probability-free characterization.…”

“…⋄ This definition essentially requires that there is no active information transmission and there is no signaling incentive. To provide an example of an information structure that is not partially nested, but that preserves convexity properties and is stochastically partially nested, let us consider the following example from [60], with the approach of this paper.…”

On the capacity of memoryless finite-state multiple access channels with asymmetric noisy state information at the encoders

“…On the other hand, using the conditional independence of from given , one gets (18) where the above inequality follows from the fact that , since removing the conditioning does not decrease the entropy, while , as is conditionally independent from given , due to the absence of output feedback. Since forms a Markov chain, the data processing inequality implies that (19) By combining (16), (17) where the last inequality follows from the independence between , and , and the fact that removing the conditioning does not decrease the entropy. Now, we have (23) where the inequality above follows from the fact that since removing the conditioning does not decrease the entropy, and that due to absence of output feedback.…”

“…Further, assume that there is no intersymbol interference, i.e., is independent from the message , and that the state process is observed through quantized observations , as discussed earlier. For the generation of optimal policies in a multiperson optimization problem, whenever a dynamic programming recursion via the construction of a Markov Chain with a fixed state space is possible (see [19] for a review of information structures in decentralized control), the optimization problem is computationally feasible and the problem is said to be tractable. In a large class of decentralized control problems, however, one faces intractable optimization problems.…”

“…We instead adopted Witsenhausen's equivalent model to generate team policies, as also elaborated in [19], by having the encoders agree on which policies to adopt before random variables are realized. The approach in our paper showed that we can obtain a direct result when the channel state sequence is memoryless.…”