The detour problem in a stochastic environment: Tolman revisited

Fakhari, Pegah; Khodadadi, Arash; Busemeyer, Jerome R.

doi:10.1016/j.cogpsych.2017.12.002

Cited by 4 publications

(5 citation statements)

References 70 publications

(127 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, the engineer may conclude that the failure was due to wear of the pump bearings and decide to replace them. Often, our actions do not lead to the desired outcome and we need to replan [1][2][3]. For instance, if changing the pump's bearings does not restore the plant's function, the engineer must determine the cause of her flawed reasoning and decide what action to take next (e.g., conduct a new test, repeat an unreliable one, or replace a different component).…”

Section: Introductionmentioning

confidence: 99%

Decision prioritization and causal reasoning in decision hierarchies

Zylberberg

2021

PLoS Comput Biol

View full text Add to dashboard Cite

From cooking a meal to finding a route to a destination, many real life decisions can be decomposed into a hierarchy of sub-decisions. In a hierarchy, choosing which decision to think about requires planning over a potentially vast space of possible decision sequences. To gain insight into how people decide what to decide on, we studied a novel task that combines perceptual decision making, active sensing and hierarchical and counterfactual reasoning. Human participants had to find a target hidden at the lowest level of a decision tree. They could solicit information from the different nodes of the decision tree to gather noisy evidence about the target’s location. Feedback was given only after errors at the leaf nodes and provided ambiguous evidence about the cause of the error. Despite the complexity of task (with 107 latent states) participants were able to plan efficiently in the task. A computational model of this process identified a small number of heuristics of low computational complexity that accounted for human behavior. These heuristics include making categorical decisions at the branching points of the decision tree rather than carrying forward entire probability distributions, discarding sensory evidence deemed unreliable to make a choice, and using choice confidence to infer the cause of the error after an initial plan failed. Plans based on probabilistic inference or myopic sampling norms could not capture participants’ behavior. Our results show that it is possible to identify hallmarks of heuristic planning with sensing in human behavior and that the use of tasks of intermediate complexity helps identify the rules underlying human ability to reason over decision hierarchies.

show abstract

Section: Introductionmentioning

confidence: 99%

Decision prioritization and causal reasoning in decision hierarchies

Zylberberg

2021

PLoS Comput Biol

View full text Add to dashboard Cite

show abstract

“…On the theoretical level, there is the question of whether cognitive maps and model-based planning are necessary for planning at all. Contemporary experimental and modelling work (Alvernhe et al 2011;Russek et al 2016;Fakhari et al 2018) suggests that they are -the performance (Alvernhe et al 2011;Fakhari et al 2018) of rats and humans in detour task experiments cannot yet be replicated (Russek et al 2016) by a purely model-free mechanism -but reinforcement learning is a fastmoving space and may advance in unexpected directions.…”

Section: Discussionmentioning

confidence: 99%

“…Extensive modelling efforts (Dolan and Dayan 2013) have validated this claim. Despite the development of sophisticated "model-free" algorithms it has thus far been impossible to replicate certain observed behaviours without using a cognitive map representation (Russek et al 2016;Fakhari et al 2018).…”

Section: Broad Rationale For Research Questionmentioning

confidence: 99%

“…Having shown that the model can perform grid world tasks, we now demonstrate that it can replicate the characteristic performance of rats running a detour maze task. This task is still considered one of the key tasks demonstrating that cognitive maps exist (Simon and Daw 2011;Russek et al 2016); work in humans strongly suggests an ability to solve detour tasks without re-learning, and these results have not yet been replicated by model-free or successor-representation based models (Russek et al 2016;Fakhari et al 2018).…”

Section: Replication Of a Detour Taskmentioning

confidence: 99%

See 1 more Smart Citation

The formation and use of hierarchical cognitive maps in the brain: A neural network model

Jordan

Navarro²,

Stringer

2020

Network: Computation in Neural Systems

View full text Add to dashboard Cite

Many researchers have tried to model how environmental knowledge is learned by the brain and used in the form of cognitive maps. However, previous work was limited in various important ways: there was little consensus on how these cognitive maps were formed and represented, the planning mechanism was inherently limited to performing relatively simple tasks, and there was little consideration of how these mechanisms would scale up. This paper makes several significant advances. Firstly, the planning mechanism used by the majority of previous work propagates a decaying signal through the network to create a gradient that points towards the goal. However, this decaying signal limited the scale and complexity of tasks that can be solved in this manner. Here we propose several ways in which a network can can self-organize a novel planning mechanism that does not require decaying activity. We also extend this model with a hierarchical planning mechanism: a layer of cells that identify frequently-used sequences of actions and reuse them to significantly increase the efficiency of planning. We speculate that our results may explain the apparent ability of humans and animals to perform model-based planning on both small and large scales without a noticeable loss of efficiency.

show abstract

“…For example, the engineer may conclude that the failure was due to wear of the pump bearings and decide to replace them. Often, our actions do not lead to the desired outcome and we need to replan (Cushing and Kambhampati, 2005;Fakhari et al, 2018;Bonet and Geffner, 2011). For instance, if changing the pump's bearings does not restore the plant's function, the engineer must determine the cause of her flawed reasoning and decide what action to take next (e.g., conduct a new test, repeat an unreliable one, or replace a different component).…”

Section: Introductionmentioning

confidence: 99%

Decision Prioritization and Causal Reasoning in Decision Hierarchies

Zylberberg¹

2021

Preprint

View full text Add to dashboard Cite

From cooking a meal to finding a route to a destination, many real life decisions can be decomposed into a hierarchy of sub-decisions. In a hierarchy, choosing which decision to think about requires planning over a potentially vast space of possible decision sequences. To gain insight into how people decide what to decide on, we studied a novel task that combines perceptual decision making, active sensing and hierarchical and counterfactual reasoning. Human participants had to find a target hidden at the lowest level of a decision tree. They could solicit information from the different nodes of the decision tree to gather noisy evidence about the target's location. Feedback was given only after errors at the leaf nodes and provided ambiguous evidence about the cause of the error. Despite the complexity of task (with $10^7$ latent states) participants were able to plan efficiently in the task. A computational model of this process identified a small number of heuristics of low computational complexity that accounted for human behavior. These heuristics include making categorical decisions at the branching points of the decision tree rather than carrying forward entire probability distributions, discarding sensory evidence deemed unreliable to make a choice, and using choice confidence to infer the cause of the error after an initial plan failed. Plans based on probabilistic inference or myopic sampling norms could not capture participants' behavior. Our results show that it is possible to identify hallmarks of heuristic planning with sensing in human behavior and that the use of tasks of intermediate complexity helps identify the rules underlying human ability to reason over decision hierarchies.

show abstract

The detour problem in a stochastic environment: Tolman revisited

Cited by 4 publications

References 70 publications

Decision prioritization and causal reasoning in decision hierarchies

Decision prioritization and causal reasoning in decision hierarchies

The formation and use of hierarchical cognitive maps in the brain: A neural network model

Decision Prioritization and Causal Reasoning in Decision Hierarchies

Contact Info

Product

Resources

About