Hierarchical motor adaptations negotiate failures during force field learning

Ikegami, Tsuyoshi; Ganesh, Gowrishankar; Gibo, Tricia L.; Yoshioka, Toshinori; Osu, Rieko; Kawato, Mitsuo

doi:10.1371/journal.pcbi.1008481

Cited by 10 publications

(4 citation statements)

References 70 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Despite the limitations, the simple linearized OFC models were crucial in determining the relationship between three properties of the task demands, its impact on coarticulation, and feedback control. Our approach is consistent with how ideal actor/estimator models have been predominantly employed as normative models in formalizing an initial hypothesis, and simulating these to generate testable predictions (29,33,(35)(36)(37)(49)(50)(51)(52)(53)(54)(55). Such normative models should be distinguished from detailed biomechanical models that can trade-off interpretability and predictability for achieving better quantitative fits.…”

Section: Limitationssupporting

confidence: 59%

Task dependent coarticulation of movement sequences

Kalidindi,

Crevecoeur

2023

Preprint

View full text Add to dashboard Cite

Classical theories propose that multiple goals of a sequence are specified simultaneously in the motor system, enabling each elements’ execution to be influenced by others in the sequence, a process known ascoarticulation. However, recent neural recordings suggest independent execution within circuits generating motor commands, with each individual goal separately specified to the controller. Consequently, the manner in which movement sequences are produced remains unclear. In a two-reaches sequence task, we tested the idea that both coarticulation and separation are related to flexible feedback control. More precisely, simulations revealed that an optimal feedback controller planning for multiple reaches at once can produce both separated and coarticulated sequences based on constraints imposed at intermediate goals. Human experiments confirmed that the coarticulation of sequential reaches was flexibly modulated by task instructions, matching simulation results closely. Moreover, the location of the second goal impacted long-latency muscle stretch responses to mechanical perturbations applied prior to the first goal, indicating that sequence combination was expressed in the sensorimotor network mediating fast feedback control. Overall, our results uncover the role of long-latency feedback pathway as a flexible control policy, supporting efficient sequence production without the need to individually specify each target one-by-one to the controller.SignificanceFrom preparing coffee to executing a tennis serve, our nervous system combines multiple elementary actions into longer sequences. Traditional research has explored whether these elements are combined in low-level control circuits for rapid execution or independently recalled from higher-level circuits. Surprisingly, the role of feedback control in the production of movement sequences has remained unexplored. Here, using simulations of optimal feedback control alongside human experiments, we shed light on the crucial role of task-dependent feedback control in shaping how sequences manifest within the motor system. Our findings reveal that the combination of sequence elements can result from flexible processing of multiple goals within the motor system involved in fast feedback control.

show abstract

Section: Limitationssupporting

confidence: 59%

Task dependent coarticulation of movement sequences

Kalidindi,

Crevecoeur

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Mistry and colleagues ( Mistry et al, 2013 ) made similar assumptions in the context of Linear-Quadratic-Gaussian (LQG) control ( Todorov and Jordan, 2002 ), with an estimated plant dynamics that differed from the true plant dynamics including the force field. Recently, Ikegami and colleagues ( Ikegami et al, 2021 ) used the same approach to demonstrate that both target failure and altered hand path may interact to drive adaptation hierarchically. In these models, the level of adaptation depended on how much the force field was compensated during movement by the approximate internal model, which simply takes the form of a function used in the controller.…”

Section: Computational Models Of Human Reaching Adaptationmentioning

confidence: 99%

Adaptive Feedback Control in Human Reaching Adaptation to Force Fields

Mathew

Crevecoeur

2021

Front. Hum. Neurosci.

View full text Add to dashboard Cite

Sensorimotor adaptation is a central function of the nervous system, as it allows humans and other animals to flexibly anticipate their interaction with the environment. In the context of human reaching adaptation to force fields, studies have traditionally separated feedforward (FF) and feedback (FB) processes involved in the improvement of behavior. Here, we review computational models of FF adaptation to force fields and discuss them in light of recent evidence highlighting a clear involvement of feedback control. Instead of a model in which FF and FB mechanisms adapt in parallel, we discuss how online adaptation in the feedback control system can explain both trial-by-trial adaptation and improvements in online motor corrections. Importantly, this computational model combines sensorimotor control and short-term adaptation in a single framework, offering novel perspectives for our understanding of human reaching adaptation and control.

show abstract

“…Hierarchical reinforcement learning was one of the strong theoretical fields with this computational objective (Wiering and Schmidhuber 1997; Parr et al 1997), and Samejima and colleagues combined hierarchical reinforcement learning with multiple internal models (Samejima et al 2003(Samejima et al , 2006Kawato and Samejima 2007). In the neuroscience of motor control, hierarchical models (Kawato et al 1987) and uniform and flat models (Todorov and Jordan 2002) were proposed, and there have been oscillations back (Scott 2004) and forth (Franklin et al 2008;Osu et al 2015;Babič et al 2016;Ikegami et al 2021). In robotics and artificial intelligence, hierarchical reinforcement learning has been explored for almost 20 years (Morimoto and Doya 2001), and recently it has regained popularity (Sugimoto et al 2012a;Merel et al 2019).…”

Section: Introductionmentioning

confidence: 99%

From internal models toward metacognitive AI

Kawato¹,

Cortese²

2021

Biol Cybern

Self Cite

View full text Add to dashboard Cite

In several papers published in Biological Cybernetics in the 1980s and 1990s, Kawato and colleagues proposed computational models explaining how internal models are acquired in the cerebellum. These models were later supported by neurophysiological experiments using monkeys and neuroimaging experiments involving humans. These early studies influenced neuroscience from basic, sensory-motor control to higher cognitive functions. One of the most perplexing enigmas related to internal models is to understand the neural mechanisms that enable animals to learn large-dimensional problems with so few trials. Consciousness and metacognition—the ability to monitor one’s own thoughts, may be part of the solution to this enigma. Based on literature reviews of the past 20 years, here we propose a computational neuroscience model of metacognition. The model comprises a modular hierarchical reinforcement-learning architecture of parallel and layered, generative-inverse model pairs. In the prefrontal cortex, a distributed executive network called the “cognitive reality monitoring network” (CRMN) orchestrates conscious involvement of generative-inverse model pairs in perception and action. Based on mismatches between computations by generative and inverse models, as well as reward prediction errors, CRMN computes a “responsibility signal” that gates selection and learning of pairs in perception, action, and reinforcement learning. A high responsibility signal is given to the pairs that best capture the external world, that are competent in movements (small mismatch), and that are capable of reinforcement learning (small reward-prediction error). CRMN selects pairs with higher responsibility signals as objects of metacognition, and consciousness is determined by the entropy of responsibility signals across all pairs. This model could lead to new-generation AI, which exhibits metacognition, consciousness, dimension reduction, selection of modules and corresponding representations, and learning from small samples. It may also lead to the development of a new scientific paradigm that enables the causal study of consciousness by combining CRMN and decoded neurofeedback.

show abstract

Hierarchical motor adaptations negotiate failures during force field learning

Cited by 10 publications

References 70 publications

Task dependent coarticulation of movement sequences

Task dependent coarticulation of movement sequences

Adaptive Feedback Control in Human Reaching Adaptation to Force Fields

From internal models toward metacognitive AI

Contact Info

Product

Resources

About