Hierarchical Behaviours: Getting the Most Bang for Your Bit

Dijk, Sander G. van; Polani, Daniel; Nehaniv, Chrystopher L.

doi:10.1007/978-3-642-21314-4_43

Cited by 13 publications

(8 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Traditionally, the formation of abstractions is conceptualized as being computationally costly because particular entities have to be grouped together by neglecting irrelevant information. Recently, abstractions that arise from sensory evolution and hierarchical behaviors have been studied from an informationtheoretic perspective (Salge and Polani, 2009;Van Dijk et al, 2011). Here, we study abstractions in the process of decisionmaking, where "similar" situations elicit the same behavior when partially ignoring the current situational context.…”

Section: Discussionmentioning

confidence: 99%

Bounded Rationality, Abstraction, and Hierarchical Decision-Making: An Information-Theoretic Optimality Principle

et al. 2015

View full text Add to dashboard Cite

Abstraction and hierarchical information processing are hallmarks of human and animal intelligence underlying the unrivaled flexibility of behavior in biological systems. Achieving such flexibility in artificial systems is challenging, even with more and more computational power. Here, we investigate the hypothesis that abstraction and hierarchical information processing might in fact be the consequence of limitations in information-processing power. In particular, we study an information-theoretic framework of bounded rational decision-making that trades off utility maximization against information-processing costs. We apply the basic principle of this framework to perception-action systems with multiple information-processing nodes and derive bounded-optimal solutions. We show how the formation of abstractions and decision-making hierarchies depends on information-processing costs. We illustrate the theoretical ideas with example simulations and conclude by formalizing a mathematically unifying optimization principle that could potentially be extended to more complex systems.

show abstract

Section: Discussionmentioning

confidence: 99%

Bounded Rationality, Abstraction, and Hierarchical Decision-Making: An Information-Theoretic Optimality Principle

et al. 2015

View full text Add to dashboard Cite

show abstract

“…In this case, however, one typically needs to include also the agent memory into the picture. For such a scenario, preliminary results indicate that informational optimality criteria have the potential to characterize general properties of information-processing architectures in a principled way (van Dijk et al 2009). …”

Section: Introductory Remarksmentioning

confidence: 99%

Information Theory of Decisions and Actions

Tishby

Polani²

2010

Perception-Action Cycle

Self Cite

220

262

View full text Add to dashboard Cite

The perception-action cycle is often defined as "the circular flow of information between an organism and its environment in the course of a sensory guided sequence of actions towards a goal" (Fuster 2001(Fuster , 2006. The question we address in this paper is in what sense this "flow of information" can be described by Shannon's measures of information introduced in his mathematical theory of communication. We provide an affirmative answer to this question using an intriguing analogy between Shannon's classical model of communication and the Perception-Action-Cycle. In particular, decision and action sequences turn out to be directly analogous to codes in communication, and their complexity -the minimal number of (binary) decisions required for reaching a goal -directly bounded by information measures, as in communication. This analogy allows us to extend the standard Reinforcement Learning framework. The latter considers the future expected reward in the course of a behaviour sequence towards a goal (value-to-go). Here, we additionally incorporate a measure of information associated with this sequence: the cumulated information processing cost or bandwidth required to specify the future decision and action sequence (information-to-go).Using a graphical model, we derive a recursive Bellman optimality equation for information measures, in analogy to Reinforcement Learning; from this, we obtain new algorithms for calculating the optimal trade-off between the value-to-go and the required information-to-go, unifying the ideas behind the Bellman and the Blahut-Arimoto iterations. This trade-off between value-to-go and information-togo provides a complete analogy with the compression-distortion trade-off in source coding. The present new formulation connects seemingly unrelated optimization problems. The algorithm is demonstrated on grid world examples.

show abstract

“…The core ideas of efficient coding theory have also been increasingly transposed into other fields, including research on memory [20,21] and language [22]. Over recent years, there have been sporadic but increasing efforts to apply the notion of efficient coding to RL models of learning and decision making [6,[23][24][25][26]. We turn now to a consideration of this enterprise, beginning with some background on RL itself.…”

Section: Botvinick and Colleaguesmentioning

confidence: 99%

Reinforcement learning, efficient coding, and the statistics of natural tasks

Botvinick

Weinstein

Solway

et al. 2015

Current Opinion in Behavioral Sciences

View full text Add to dashboard Cite

The application of ideas from computational reinforcement learning has recently enabled dramatic advances in behavioral and neuroscientific research. For the most part, these advances have involved insights concerning the algorithms underlying learning and decision making. In the present article, we call attention to the equally important but relatively neglected question of how problems in learning and decision making are internally represented. To articulate the significance of representation for reinforcement learning we draw on the concept of efficient coding, as developed in perception research. The resulting perspective exposes a range of novel goals for behavioral and neuroscientific research, highlighting in particular the need for research into the statistical structure of naturalistic tasks.

show abstract

Hierarchical Behaviours: Getting the Most Bang for Your Bit

Cited by 13 publications

References 14 publications

Bounded Rationality, Abstraction, and Hierarchical Decision-Making: An Information-Theoretic Optimality Principle

Bounded Rationality, Abstraction, and Hierarchical Decision-Making: An Information-Theoretic Optimality Principle

Information Theory of Decisions and Actions

Reinforcement learning, efficient coding, and the statistics of natural tasks

Contact Info

Product

Resources

About