Risk-Sensitivity in Sensorimotor Control

Braun, Daniel A.; Nagengast, Arne J.; Wolpert, Daniel M.

doi:10.3389/fnhum.2011.00001

Cited by 194 publications

(296 citation statements)

References 87 publications

Supporting

Mentioning

276

Contrasting

Order By: Relevance

“…Therefore, choosing how to move an effector can also be considered as a decision-making problem [72]. While many previous studies have investigated the optimization of expected movement costs to describe behaviour, a number of recent studies has found risk-sensitive deviations from expected utility theory in sensorimotor control [73][74][75][76][77].…”

Section: (D) Application To Sensorimotor Controlmentioning

confidence: 99%

Thermodynamics as a theory of decision-making with information-processing costs

2013

Self Cite

View full text Add to dashboard Cite

Perfectly rational decision-makers maximize expected utility, but crucially ignore the resource costs incurred when determining optimal actions. Here, we propose a thermodynamically inspired formalization of bounded rational decision-making where information processing is modelled as state changes in thermodynamic systems that can be quantified by differences in free energy. By optimizing a free energy, bounded rational decision-makers trade off expected utility gains and information-processing costs measured by the relative entropy. As a result, the bounded rational decision-making problem can be rephrased in terms of well-known variational principles from statistical physics. In the limit when computational costs are ignored, the maximum expected utility principle is recovered. We discuss links to existing decision-making frameworks and applications to human decision-making experiments that are at odds with expected utility theory. Since most of the mathematical machinery can be borrowed from statistical physics, the main contribution is to reinterpret the formalism of thermodynamic free-energy differences in terms of bounded rational decisionmaking and to discuss its relationship to human decision-making experiments.

show abstract

Section: (D) Application To Sensorimotor Controlmentioning

confidence: 99%

Thermodynamics as a theory of decision-making with information-processing costs

2013

Self Cite

View full text Add to dashboard Cite

show abstract

“…Recently, it was found that risk sensitivity is an important determinant in human sensorimotor behaviour [38]. Risksensitive decision-makers do not base their choices exclusively on the expectation value of a particular cost function, but they also consider higher-order moments of this cost function.…”

Section: Discussionmentioning

confidence: 99%

The effect of model uncertainty on cooperation in sensorimotor interactions

Grau-Moya

Hez

Pezzulo³

et al. 2013

J. R. Soc. Interface.

View full text Add to dashboard Cite

Decision-makers have been shown to rely on probabilistic models for perception and action. However, these models can be incorrect or partially wrong in which case the decision-maker has to cope with model uncertainty. Model uncertainty has recently also been shown to be an important determinant of sensorimotor behaviour in humans that can lead to risk-sensitive deviations from Bayes optimal behaviour towards worst-case or best-case outcomes. Here, we investigate the effect of model uncertainty on cooperation in sensorimotor interactions similar to the stag-hunt game, where players develop models about the other player and decide between a pay-off-dominant cooperative solution and a risk-dominant, non-cooperative solution. In simulations, we show that players who allow for optimistic deviations from their opponent model are much more likely to converge to cooperative outcomes. We also implemented this agent model in a virtual reality environment, and let human subjects play against a virtual player. In this game, subjects' pay-offs were experienced as forces opposing their movements. During the experiment, we manipulated the risk sensitivity of the computer player and observed human responses. We found not only that humans adaptively changed their level of cooperation depending on the risk sensitivity of the computer player but also that their initial play exhibited characteristic risksensitive biases. Our results suggest that model uncertainty is an important determinant of cooperation in two-player sensorimotor interactions.

show abstract

“…The optimal control solution is given by the control law u * (·) that minimizes (3) constrained to dynamics (5) and (4). Due to the stochastic nature of (5), the cost to be optimized is a random variable.…”

Section: Problem Setting and Approachmentioning

confidence: 99%

“…The standard treatment in these settings is to minimize the expected value of the resulting stochastic cost. In contrast, human decisions under variability have been more accurately modeled by risk-sensitive decision-makers [4], which also considers cost variance. These results motivate us to consider high order cost statistics for control in robotic applications.…”

Section: Introductionmentioning

confidence: 99%

Uncertainty-dependent optimal control for robot control considering high-order cost statistics

Medina

Hirche

2015

2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

Abstract-As the application of probabilistic models in robotic applications increases, the necessity of a systematic robot-control method that considers the effects of multiple uncertainty sources becomes more evident. Motivated by human sensorimotor findings, in this work we study the stochastic locally optimal feedback control problem with high-order cost statistics where dynamics have multiple additive noise sources and cost variability produced by each uncertainty source is evaluated marginally. We present risk-sensitive and costcumulant solutions for this problem for non-linear dynamics and non-quadratic costs. Locally optimal solutions are found by iteratively performing a linear quadratic approximation around a nominal trajectory, solving the local problem and updating the trajectory until convergence. Simulation results of a point mass robot and a two-link manipulator validate the applicability of the proposed approach and illustrate its peculiarities.

show abstract

Risk-Sensitivity in Sensorimotor Control

Cited by 194 publications

References 87 publications

Thermodynamics as a theory of decision-making with information-processing costs

Thermodynamics as a theory of decision-making with information-processing costs

The effect of model uncertainty on cooperation in sensorimotor interactions

Uncertainty-dependent optimal control for robot control considering high-order cost statistics

Contact Info

Product

Resources

About