Combining Electromyography and Fiducial Marker Based Tracking for Intuitive Telemanipulation with a Robot Arm Hand System

Dwivedi, Anany; Gorjup, Gal; Kwon, Yongje; Liarokapis, Minas

doi:10.1109/ro-man46459.2019.8956456

Cited by 20 publications

(9 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One of the main potential applications of MCSs is powered upper‐limb prostheses and electric‐powered wheelchairs. Dwivedi et al [ 30 ] used EMG and fiducial marker‐based tracking to capture the myoelectric activations of the user during the execution of specific hand gestures. The device demonstrated to be capable of controlling a dexterous robot arm hand system and translating the gestures into the desired grasp type for the robot hand.…”

Section: Sensing Modalitiesmentioning

confidence: 99%

A Review of Hand Gesture Recognition Systems Based on Noninvasive Wearable Sensors

Tchantchane,

Zhou,

Zhang

et al. 2023

Advanced Intelligent Systems

View full text Add to dashboard Cite

Hand gesture, one of the essential ways for a human to convey information and express intuitive intention, has a significant degree of differentiation, substantial flexibility, and high robustness of information transmission to make hand gesture recognition (HGR) one of the research hotspots in the fields of human–human and human–computer or human–machine interactions. Noninvasive, on‐body sensors can monitor, track, and recognize hand gestures for various applications such as sign language recognition, rehabilitation, myoelectric control for prosthetic hands and human–machine interface (HMI), and many other applications. This article systematically reviews recent achievements from noninvasive upper‐limb sensing techniques for HGR, multimodal sensing fusion to gain additional user information, and wearable gesture recognition algorithms to obtain more reliable and robust performance. Research challenges, progress, and emerging opportunities for sensor‐based HGR systems are also analyzed to provide perspectives for future research and progress.

show abstract

Section: Sensing Modalitiesmentioning

confidence: 99%

A Review of Hand Gesture Recognition Systems Based on Noninvasive Wearable Sensors

Tchantchane,

Zhou,

Zhang

et al. 2023

Advanced Intelligent Systems

View full text Add to dashboard Cite

show abstract

“…The mean squared error (MSE) loss function was employed during training. The MSE is defined as follows l(y, ŷ) = (y − ŷ) 2 (1…”

Section: Training and Evaluationmentioning

confidence: 99%

“…These signals measure the myoelectric activations of the human muscles generated during contraction and offer an intuitive method for developing HMIs. EMG-based interfaces can decode human movement intention to classify hand gestures and motions [1], [2], as well as the execution of in-hand manipulation motions 1 Ricardo V. Godoy and Minas Liarokapis are with the New Dexterity research group, Department of Mechanical and Mechatronics Engineering, The University of Auckland, New Zealand. E-mails: rdeg264@aucklanduni.ac.nz, minas.liarokapis@auckland.ac.nz 2 Anany Dwivedi is with the Chair of Autonomous Systems and Mechatronics, Friedrich-Alexander-Universit ät Erlangen-N ürnberg, Germany.…”

Section: Introductionmentioning

confidence: 99%

Electromyography Based Decoding of Dexterous, In-Hand Manipulation Motions With Temporal Multichannel Vision Transformers

Godoy

Dwivedi

Liarokapis

2022

IEEE Trans. Neural Syst. Rehabil. Eng.

Self Cite

View full text Add to dashboard Cite

Electromyography (EMG) signals have been used in designing muscle-machine interfaces (MuMIs) for various applications, ranging from entertainment (EMG controlled games) to human assistance and human augmentation (EMG controlled prostheses and exoskeletons). For this, classical machine learning methods such as Random Forest (RF) models have been used to decode EMG signals. However, these methods depend on several stages of signal pre-processing and extraction of handcrafted features so as to obtain the desired output. In this work, we propose EMG based frameworks for the decoding of object motions in the execution of dexterous, inhand manipulation tasks using raw EMG signals input and two novel deep learning (DL) techniques called Temporal Multi-Channel Transformers and Vision Transformers. The results obtained are compared, in terms of accuracy and speed of decoding the motion, with RF-based models and Convolutional Neural Networks as a benchmark. The models are trained for 11 subjects in a motion-object specific and motion-object generic way, using the 10-fold crossvalidation procedure. This study shows that the performance of MuMIs can be improved by employing DL-based models with raw myoelectric activations instead of developing DL or classic machine learning models with handcrafted features.

show abstract

“…Machine learning (ML) techniques have been employed to decode EMG signals to perform both classification (e.g. decoding discrete human gestures) [5]- [8] and regression (e.g. decoding continuous human motions) [9]- [11].…”

Section: Introductionmentioning

confidence: 99%

On EMG Based Dexterous Robotic Telemanipulation: Assessing Machine Learning Techniques, Feature Extraction Methods, and Shared Control Schemes

et al. 2022

Self Cite

View full text Add to dashboard Cite

Electromyography (EMG) signals are commonly used for the development of Muscle Machine Interfaces. EMG-based solutions provide intuitive and often hand-free control in a wide range of applications that range from the decoding of human intention in classification tasks to the continuous decoding of human motion employing regression models. In this work, we compare various machine learning and feature extraction methods for the creation of EMG based control frameworks for dexterous robotic telemanipulation. Various models are needed that can decode dexterous, in-hand manipulation motions and perform hand gesture classification in real-time. Three different machine learning methods and eight different time-domain features were evaluated and compared. The performance of the models was evaluated in terms of accuracy and time required to predict a data sample. The model that presented the best performance and prediction time trade-off was used for executing in real-time a telemanipulation task with the New Dexterity Autonomous Robotic Assistance (ARoA) platform (a humanoid robot). Various experiments have been conducted to experimentally validate the efficiency of the proposed methods. The robotic system is shown to successfully complete a series of tasks autonomously as well as to efficiently execute tasks in a shared control manner.

show abstract

Combining Electromyography and Fiducial Marker Based Tracking for Intuitive Telemanipulation with a Robot Arm Hand System

Cited by 20 publications

References 25 publications

A Review of Hand Gesture Recognition Systems Based on Noninvasive Wearable Sensors

A Review of Hand Gesture Recognition Systems Based on Noninvasive Wearable Sensors

Electromyography Based Decoding of Dexterous, In-Hand Manipulation Motions With Temporal Multichannel Vision Transformers

On EMG Based Dexterous Robotic Telemanipulation: Assessing Machine Learning Techniques, Feature Extraction Methods, and Shared Control Schemes

Contact Info

Product

Resources

About