Information-Theoretic Bounded Rationality and ε-Optimality

Abstract

Bounded rationality concerns the study of decision makers with limited information processing resources. Previously, the free energy difference functional has been suggested to model bounded rational decision making, as it provides a natural trade-off between an energy or utility function that is to be optimized and information processing costs that are measured by entropic search costs. The main question of this article is how the information-theoretic free energy model relates to simple -optimality models of bounded rational decision making, where the decision maker is satisfied with any action in an -neighborhood of the optimal utility. We find that the stochastic policies that optimize the free energy trade-off comply with the notion of -optimality. Moreover, this optimality criterion even holds when the environment is adversarial. We conclude that the study of bounded rationality based on -optimality criteria that abstract away from the particulars of the information processing constraints is compatible with the information-theoretic free energy model of bounded rationality.

Bounded Rationality, Abstraction, and Hierarchical Decision-Making: An Information-Theoretic Optimality Principle

“…In this study, we use an information-theoretic model of bounded rational decision-making Braun, 2012, 2013;Braun and Ortega, 2014; that has precursors in the economic literature (McKelvey and Palfrey, 1995;Mattsson and Weibull, 2002;Sims, 2003Sims, , 2005Sims, , 2006Sims, , 2010Wolpert, 2006) and that is closely related to recent advances in the information theory of perception-action systems (Todorov, 2007(Todorov, , 2009Still, 2009;Friston, 2010;Peters et al, 2010;Tishby and Polani, 2011;Daniel et al, 2012Daniel et al, , 2013Kappen et al, 2012;Rawlik et al, 2012;Rubin et al, 2012;Neymotin et al, 2013;Tkačik and Bialek, 2014;Palmer et al, 2015). The basis of this approach is formalized by a free energy principle that trades off expected utility, and the cost of computation that is required to adapt the system accordingly in order to achieve high utility.…”

Non-Equilibrium Relations for Bounded Rational Decision-Making in Changing Environments

“…Originally proposed by Herbert Simon [ 7 , 8 ], bounded rationality comprises a medley of approaches ranging from optimization-based approaches like bounded optimality (searching for the program that achieves the best utility performance on a particular platform) [ 9 , 10 , 11 ] and meta-reasoning (optimizing the cost of reasoning) [ 12 , 13 , 14 ] to heuristic approaches that reject the notion of optimization [ 15 , 16 , 17 ]. Recently, new impulses for the development of bounded rationality theory have come from information-theoretic and thermodynamic perspectives on the general organization of perception-action-systems [ 1 , 3 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 ]. In the economic and game-theoretic literature, these models have precursors that have studied bounded rationality inspired by stochastic choice rules originally proposed by Luce, McFadden and others [ 2 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 ].…”

An information-theoretic on-line update principle for perception-action coupling

“…In this study, we use an information-theoretic model of bounded rational decision-making [19], [20], [21], [22], [23]. In a decision-making task with context, an agent is presented with a world-state ω and has to find an optimal action a * ω from a set of admissible actions.…”