Deep Neural Network Approach in EMG-Based Force Estimation for Human–Robot Interaction

Su, Hang; Qi, Wen; Li, Zhijun; Chen, Ziyang; Ferrigno, Giancarlo; Momi, Elena De

doi:10.1109/tai.2021.3066565

Cited by 55 publications

(15 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Rane et al [21] employed a deep neural network to learn the feature mapping from movement space to muscle space, so musculoskeletal force could be predicted from kinematics. Similar ideas were also reported in [22]- [25]. However, data-driven models are established without explicit physics modelling of the underlying neuromechanical processes, and they are essentially "black-box" tools where all intermediate functional relationships cannot reflect the mechanisms underlying the observed variables [26], [27].…”

Section: Introductionmentioning

confidence: 56%

Physics-informed Deep Learning for Musculoskeletal Modelling: Predicting Muscle Forces and Joint Kinematics from Surface EMG

Zhang¹,

Zhao²,

Shone³

et al. 2022

Preprint

View full text Add to dashboard Cite

Musculoskeletal models have been widely used for detailed biomechanical analysis to characterise various functional impairments given their ability to estimate movement variables (i.e., muscle forces and joint moment) which cannot be readily measured in vivo. Physics-based computational neuromusculoskeletal models can interpret the dynamic interaction between neural drive to muscles, muscle dynamics, body and joint kinematics and kinetics. Still, such set of solutions suffers from slowness, especially for the complex models, hindering the utility in real-time applications. In recent years, datadriven methods has emerged as a promising alternative due to the benefits in speedy and simple implementation, but they cannot reflect the underlying neuromechanical processes. This paper proposes a physics-informed deep learning framework for musculoskeletal modelling, where physics-based domain knowledge is brought into the data-driven model as soft constraints to penalise/regularise the data-driven model. We use the synchronous muscle forces and joint kinematics prediction from surface electromyogram (sEMG) as the exemplar to illustrate the proposed framework. Convolutional neural network (CNN) is employed as the deep neural network to implement the proposed framework. At the same time, the physics law between muscle forces and joint kinematics is used the soft constraint. Experimental validations on two groups of data, including one benchmark dataset and one self-collected dataset from six healthy subjects, are performed. The experimental results demonstrate the effectiveness and robustness of the proposed framework.

show abstract

Section: Introductionmentioning

confidence: 56%

Physics-informed Deep Learning for Musculoskeletal Modelling: Predicting Muscle Forces and Joint Kinematics from Surface EMG

Zhang¹,

Zhao²,

Shone³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In our future work, sensor fusion and deep learning will be used to recognize human activities (Qi et al, 2021a;Qi et al, 2021b). Complex environments and case studies will be considered to make the solutions more reliable and natural (Su et al, 2021a(Su et al, , 2021b; Chen and Qiao (2020a).…”

Section: Conclusion and Discussionmentioning

confidence: 99%

“…In general, hands-on control and teleoperation are two conventional human operational activities in the surgical operating room. "Handson": the robot is compliant and can be moved by the operator's hand (Beretta et al, 2015;Su et al, 2020aSu et al, , 2020b; "Teleoperation": the movement of the robot manipulator is driven through the remote device with a planned mapping (Su et al, 2021a(Su et al, , 2021b. Both procedures are activated and their corresponding controllers switch based on the actual human activity (Qi et al, 2020) in the surgical operations, for example, the neuroArm (Sutherland et al, 2008), the Da Vinci robot (Guthart and Salisbury, 2000) and the MiroSurge (Hagn et al, 2010).…”

Section: Introductionmentioning

confidence: 99%

A human activity-aware shared control solution for medical human–robot interaction

Schmirander

et al. 2022

Self Cite

View full text Add to dashboard Cite

Purpose The purpose of this paper is to develop a human activity-aware adaptive shared control solution for human–robot interaction in surgical operation. Hands-on control and teleoperation are two main procedures switched frequently in teleoperated minimally invasive surgery (MIS). The detailed human activity in the procedures can be defined and recognized using the sensor information. In this paper, a novel continuous adaptive shared control method is proposed for manipulators with Cartesian impedance control in the surgical scenario. Design/methodology/approach A human activity-aware shared control solution by adjusting the weight function is introduced to achieve smooth transition among different human activities, including hands-on control and teleoperation. Instead of introducing various controllers and switching among them during the surgical procedures, the proposed solution integrated all the human activity-based controllers into a single controller and the transition among the procedures is smooth and stable. The effectiveness of the proposed control approach was verified in a lab setup environment. The results prove that the robot behavior is stable and smooth. The algorithm is feasible and can achieve a human activity-aware adaptive shared control solution for human–robot interaction in surgical operation. Findings Based on the experiment, the results confirm that the proposed human activity-aware adaptive shared control solution can switch the device behavior automatically using the real-time sensor information. The transition between different activities is smooth and stable. Practical implications For teleoperated surgical applications, the proposed method integrated different controllers for various human activities into a single controller by recognizing the activities using the real-time sensor information and the transition between different procedures is smooth and stable. It eases the surgical work for the surgeon and enhances the safety during the transition of control modes. The presented scheme provides a general solution to address the switching of working procedures in teleoperated MIS. Originality/value To the best of the authors’ knowledge, this paper is the first to propose human activity-aware adaptive shared control solution for human–robot interaction in surgical operations.

show abstract

“…For instance, EMG signals can be used to estimate human motion in 3D space, with recurrent neural network architectures yielding better results than a support vector regression (SVR) or a non-recurrent network architecture [50]. In [51], a deep convolutional neural network was used to estimate interaction force from EMG, potentially avoiding the need to incorporate interaction force sensors to facilitate human-robot physical interaction. However, prediction based on EMG signals will suffer from their large inter-trials variability and noise [26].…”

Section: A Motion Intent Estimationmentioning

confidence: 99%

A review on interaction control for contact robots through intent detection

Sena

Wang

et al. 2022

Prog. Biomed. Eng.

View full text Add to dashboard Cite

Interaction control can take opportunities offered by contact robots physically interacting with their human user, such as assistance targeted to each human user, communication of goals to enable effective teamwork, and task-directed motion resistance in physical training and rehabilitation contexts. Here we review the burgeoning field of interaction control in the control theory and machine learning communities, by analysing the exchange of haptic information between the robot and its human user, and how they share the task effort. We first review the estimation and learning methods to predict the human user intent with the large uncertainty, variability and noise and limited observation of human motion. Based on this motion intent core, typical interaction control strategies are described using a homotopy of shared control parameters. Recent methods of haptic communication and game theory are then presented to consider the co-adaptation of human and robot control and yield versatile interactive control as observed between humans. Finally, the limitations of the presented state of the art are discussed and directions for future research are outlined.

show abstract

Deep Neural Network Approach in EMG-Based Force Estimation for Human–Robot Interaction

Cited by 55 publications

References 42 publications

Physics-informed Deep Learning for Musculoskeletal Modelling: Predicting Muscle Forces and Joint Kinematics from Surface EMG

Physics-informed Deep Learning for Musculoskeletal Modelling: Predicting Muscle Forces and Joint Kinematics from Surface EMG

A human activity-aware shared control solution for medical human–robot interaction

A review on interaction control for contact robots through intent detection

Contact Info

Product

Resources

About