Rationally Inattentive Control of Markov Processes

Shafieepoorfard, Ehsan; Raginsky, Maxim; Meyn, Sean

doi:10.1137/15m1008476

Cited by 23 publications

(28 citation statements)

References 36 publications

(49 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The problem of minimizing an arbitrary cost function in control of a general process under a directed mutual information constraint was formulated in [26]. Control of general Markov processes under a mutual information constraint was studied in [27]. Silva et al [28] elucidated the operational meaning of directed mutual information, by pointing out that it lower-bounds the rate of a quantizer embedded into a feedback loop of a control system, and by showing that the bound is approached to within 1 bit by a dithered prefix-free quantizer, a compression setting in which both the compressor and the decompressor have access to a common dither -a random signal with special statistical properties.…”

Section: Prior Artmentioning

confidence: 99%

Rate-Cost Tradeoffs in Control

Kostina

Hassibi

2019

IEEE Trans. Automat. Contr.

112

158

View full text Add to dashboard Cite

Consider a control problem with a communication channel connecting the observer of a linear stochastic system to the controller. The goal of the controller is to minimize a quadratic cost function in the state variables and control signal, known as the linear quadratic regulator (LQR). We study the fundamental tradeoff between the communication rate r bits/sec and the expected cost b. We obtain a lower bound on a certain rate-cost function, which quantifies the minimum directed mutual information between the channel input and output that is compatible with a target LQR cost. The rate-cost function has operational significance in multiple scenarios of interest: among others, it allows us to lower-bound the minimum communication rate for fixed and variable length quantization, and for control over noisy channels. We derive an explicit lower bound to the rate-cost function, which applies to the vector, non-Gaussian, and partially observed systems, thereby extending and generalizing an earlier explicit expression for the scalar Gaussian system, due to Tatikonda el al. [2]. The bound applies as long as the differential entropy of the system noise is not −∞. It can be closely approached by a simple lattice quantization scheme that only quantizes the innovation, that is, the difference between the controller's belief about the current state and the true state. Via a separation principle between control and communication, similar results hold for causal lossy compression of additive noise Markov sources. Apart from standard dynamic programming arguments, our technical approach leverages the Shannon lower bound, develops new estimates for data compression with coding memory, and uses some recent results on high resolution variablelength vector quantization to prove that the new converse bounds are tight.

show abstract

Section: Prior Artmentioning

confidence: 99%

Rate-Cost Tradeoffs in Control

Kostina

Hassibi

2019

IEEE Trans. Automat. Contr.

112

158

View full text Add to dashboard Cite

show abstract

“…It is surprising that an MDP can be solved using an ODE under general conditions, and fortunate that this ODE admits simple structure in the K-L cost framework that is a focus of the paper. It is likely that the ODE has special structure for other classes of MDPs, such as the "rational inattention" framework of [21,22,19,20]. The computational efficiency of this approach will depend in part on numerical properties of the ODE, such as its sensitivity for high dimensional models.…”

Section: Discussionmentioning

confidence: 99%

“…It is called a "rate function" because it defines the relative entropy rate between two stationary Markov chains, and appears in the theory of large deviations for Markov chains [12]. For the transition matrix P h defined in (19), the rate function can be expressed in terms of its invariant pmf π h , the bivariate pmf Π h (x, x ) = π h (x)P h (x, x ), and the log moment generating function (20):…”

Section: Assumptions and Notationmentioning

confidence: 99%

Ordinary Differential Equation Methods for Markov Decision Processes and Application to Kullback--Leibler Control Cost

Bušíć

Meyn²

2018

SIAM J. Control Optim.

View full text Add to dashboard Cite

A new approach to computation of optimal policies for MDP (Markov decision process) models is introduced. The main idea is to solve not one, but an entire family of MDPs, parameterized by a scalar ζ that appears in the one-step reward function. For an MDP with d states, the family of value functions {h * ζ : ζ ∈ R} is the solution to an ODE,where the vector field V : R d → R d has a simple form, based on a matrix inverse. This general methodology is applied to a family of average-cost optimal control models in which the one-step reward function is defined by Kullback-Leibler divergence. The motivation for this reward function in prior work is computation: The solution to the MDP can be expressed in terms of the Perron-Frobenius eigenvector for an associated positive matrix. The drawback with this approach is that no hard constraints on the control are permitted. It is shown here that it is possible to extend this framework to model randomness from nature that cannot be modified by the controller. Perron-Frobenius theory is no longer applicable -the resulting dynamic programming equations appear as complex as a completely unstructured MDP model. Despite this apparent complexity, it is shown that this class of MDPs admits a solution via this ODE technique. This approach is new and practical even for the simpler problem in which randomness from nature is absent.

show abstract

“…Consumers choose to be rationally inattentive indicating a perceived trade‐off between the costs of procuring and the expected benefits from additional information (Sallee, 2014). Inattention in the behavioral economic literature suggests that the resources necessary to calculate the payoff from attention to information (i.e., time and money) are commonly perceived to be too costly, often exceeding what the typical consumer is able or willing to invest (Shafieepoorfard, Raginsky, & Meyn, 2013). Consumer inattention varies, falling somewhere between the two extremes––completely attentive and completely inattentive; these choices vary given the context of the situation, being partially dependent upon characteristics of the consumer or aspects within the choice itself (Palmer & Walls, 2015).…”

Section: Literature Reviewmentioning

confidence: 99%

Susceptibility to Inattention: Unpacking Who is Susceptible to Inattention in Energy‐Based Electronic Billing

Curley

Rustamov

Harrison

et al. 2020

Review of Policy Research

View full text Add to dashboard Cite

In this paper, we examine characteristics that may change the susceptibility to inattention in electronic billing (e-billing). Digitization of energy bills can increase the delivery of energy feedback and increase knowledge around conservation efforts, only when attention remains at similar levels to that of paper bills. We hypothesize that only subsets of the population are susceptible to inattention in e-billing. We do this by estimating energy consumption for e-bill and paper billers controlling for several characteristics of participants, homes, and weather in the City of Tallahassee, Florida. We use a difference-indifferences (DD) approach to estimate the effects of the e-bill participation, which is a common approach for observational and quasiexperimental settings. We find that budget constraints limit an individual's susceptibility to inattention in e-billing, with lower income groups decreasing energy consumption on average by 4.4% but has no effect on higher income groups. This suggests that inattention may not occur at the same levels or for the same reasons for all members of the public. This has implications regarding the practice of policy design and communication strategies for the public at large.

show abstract

Rationally Inattentive Control of Markov Processes

Cited by 23 publications

References 36 publications

Rate-Cost Tradeoffs in Control

Rate-Cost Tradeoffs in Control

Ordinary Differential Equation Methods for Markov Decision Processes and Application to Kullback--Leibler Control Cost

Susceptibility to Inattention: Unpacking Who is Susceptible to Inattention in Energy‐Based Electronic Billing

Contact Info

Product

Resources

About