A hierarchical sensorimotor control framework for human-in-the-loop robotic hands

Seminara, Lucia; Došen, Strahinja; Mastrogiovanni, Fulvio; Bianchi, Matteo; Watt, Simon J.; Beckerle, Philipp; Nanayakkara, Thrishantha; Drewing, Knut; Moscatelli, Alessandro; Klatzky, Roberta L.; Loeb, Gerald E.

doi:10.1126/scirobotics.add5434

Cited by 12 publications

(7 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hierarchical control strategies are based on the idea that the brain controls the hand by using different levels of abstraction and representation, from high-level goals and intentions to low-level motor commands and signals [48,[89][90][91][92], and can provide a framework for mapping between different levels of control and representation [93], as well as for integrating multiple sources of information, such as visual, proprioceptive, tactile, and emotional cues [94,95].…”

Section: Hierarchical Control Strategies For Prosthetic Handsmentioning

confidence: 99%

A Perspective on Prosthetic Hands Control: From the Brain to the Hand

Gentile,

Gruppioni

2023

Prosthesis

View full text Add to dashboard Cite

The human hand is a complex and versatile organ that enables humans to interact with the environment, communicate, create, and use tools. The control of the hand by the brain is a crucial aspect of human cognition and behaviour, but also a challenging problem for both neuroscience and engineering. The aim of this study is to review the current state of the art in hand and grasp control from a neuroscientific perspective, focusing on the brain mechanisms that underlie sensory integration for hand control and the engineering implications for developing artificial hands that can mimic and interface with the human brain. The brain controls the hand by processing and integrating sensory information from vision, proprioception, and touch, using different neural pathways. The user’s intention can be obtained to control the artificial hand by using different interfaces, such as electromyography, electroneurography, and electroencephalography. This and other sensory information can be exploited by different learning mechanisms that can help the user adapt to changes in sensory inputs or outputs, such as reinforcement learning, motor adaptation, and internal models. This work summarizes the main findings and challenges of each aspect of hand and grasp control research and highlights the gaps and limitations of the current approaches. In the last part, some open questions and future directions for hand and grasp control research are suggested by emphasizing the need for a neuroscientific approach that can bridge the gap between the brain and the hand.

show abstract

Section: Hierarchical Control Strategies For Prosthetic Handsmentioning

confidence: 99%

A Perspective on Prosthetic Hands Control: From the Brain to the Hand

Gentile,

Gruppioni

2023

Prosthesis

View full text Add to dashboard Cite

show abstract

“…Endowing robots flexibly interact with changing and unpredictable environments toward the completion of specific tasks is one of the goals in embodied intelligence 1 , 2 . The progress of deep learning algorithms and computing units promotes the field towards such an aspiration 3 – 7 . However, current intelligent robots remain incapable of performing as well as humans or animals, even in some reflex-like behaviors, such as object grasping or emergency escape.…”

Section: Introductionmentioning

confidence: 99%

Firing feature-driven neural circuits with scalable memristive neurons for robotic obstacle avoidance

Yang,

Zhu,

Zhang

et al. 2024

Nat Commun

View full text Add to dashboard Cite

Neural circuits with specific structures and diverse neuronal firing features are the foundation for supporting intelligent tasks in biology and are regarded as the driver for catalyzing next-generation artificial intelligence. Emulating neural circuits in hardware underpins engineering highly efficient neuromorphic chips, however, implementing a firing features-driven functional neural circuit is still an open question. In this work, inspired by avoidance neural circuits of crickets, we construct a spiking feature-driven sensorimotor control neural circuit consisting of three memristive Hodgkin-Huxley neurons. The ascending neurons exhibit mixed tonic spiking and bursting features, which are used for encoding sensing input. Additionally, we innovatively introduce a selective communication scheme in biology to decode mixed firing features using two descending neurons. We proceed to integrate such a neural circuit with a robot for avoidance control and achieve lower latency than conventional platforms. These results provide a foundation for implementing real brain-like systems driven by firing features with memristive neurons and put constructing high-order intelligent machines on the agenda.

show abstract

“…Robotic manipulators, for example, are crucial for a variety of applications serving in versatile and dynamic environments, ranging from industrial assembly lines to neural prostheses. Highly adaptive and localized control close to the sensory nodes can drastically improve performance and can also warrant operational safety which is essential for human-oriented purposes such as neuroprosthetics 38 , 39 .…”

Section: Introductionmentioning

confidence: 99%

Bio-inspired multimodal learning with organic neuromorphic electronics for behavioral conditioning in robotics

Krauhausen,

Griggs,

McCulloch

et al. 2024

Nat Commun

View full text Add to dashboard Cite

Biological systems interact directly with the environment and learn by receiving multimodal feedback via sensory stimuli that shape the formation of internal neuronal representations. Drawing inspiration from biological concepts such as exploration and sensory processing that eventually lead to behavioral conditioning, we present a robotic system handling objects through multimodal learning. A small-scale organic neuromorphic circuit locally integrates and adaptively processes multimodal sensory stimuli, enabling the robot to interact intelligently with its surroundings. The real-time handling of sensory stimuli via low-voltage organic neuromorphic devices with synaptic functionality forms multimodal associative connections that lead to behavioral conditioning, and thus the robot learns to avoid potentially dangerous objects. This work demonstrates that adaptive neuro-inspired circuitry with multifunctional organic materials, can accommodate locally efficient bio-inspired learning for advancing intelligent robotics.

show abstract

A hierarchical sensorimotor control framework for human-in-the-loop robotic hands

Cited by 12 publications

References 33 publications

A Perspective on Prosthetic Hands Control: From the Brain to the Hand

A Perspective on Prosthetic Hands Control: From the Brain to the Hand

Firing feature-driven neural circuits with scalable memristive neurons for robotic obstacle avoidance

Bio-inspired multimodal learning with organic neuromorphic electronics for behavioral conditioning in robotics

Contact Info

Product

Resources

About