Feasibility of Wrist-Worn, Real-Time Hand, and Surface Gesture Recognition via sEMG and IMU Sensing

Jiang, Shuo; Lv, Bo; Guo, Weichao; Zhang, Chao; Wang, Haitao; Sheng, Xinjun; Shull, Peter B.

doi:10.1109/tii.2017.2779814

Cited by 181 publications

(110 citation statements)

References 33 publications

Supporting

Mentioning

105

Contrasting

Order By: Relevance

“…Hand gesture recognition enables intuitive interaction with electronic and mechanical systems such as smart phone and prosthetic hand [1], [2]. Pattern recognition (PR) technique has been widely researched in this field, and desirable classification accuracy has been achieved using different sensing modalities such as surface electromyography (sEMG) [3], ultrasound [4], computer vision [5], etc.…”

Section: Introductionmentioning

confidence: 99%

A Proportional Pattern Recognition Control Scheme for Wearable A-mode Ultrasound Sensing

Yang

Yan

Zhenfeng

et al. 2020

IEEE Trans. Ind. Electron.

View full text Add to dashboard Cite

It is evident that the prevailing solution, myoelectric pattern recognition for prosthetic manipulation, constrains gesture-based interaction due to the lack of proportional control information such as exerted force. This paper reports an attempt, named simultaneous gesture recognition and muscle contraction force estimation, to realize proportional pattern recognition (PPR) control taking advantage of arm muscle deformation via wearable ultrasound sensing. We experiment with eight types of predefined hand motions, with a range of 0 -60% maximum voluntary contraction (MVC) using a wearable multi-channel Amode ultrasound system. The experiment result demonstrates that above 93.7% of gestures are correctly recognized during dynamic muscle contraction forces (0 -60% MVC), albeit only training at a slight force level (<6% MVC). Besides, the adopted non-parametric Gaussian process regression estimates the muscle contraction forces accurately synchronously, with average coefficient of determination (R 2 ) and normalized root-mean-square error (nRMSE) of 0.927 and 0.102, respectively. These research outcomes demonstrate the feasibility of ultrasound-based PPR control, paving the way for musculature-driven applications including finer prosthetic control, remote manipulation and rehabilitation treatment.

show abstract

Section: Introductionmentioning

confidence: 99%

A Proportional Pattern Recognition Control Scheme for Wearable A-mode Ultrasound Sensing

Yang

Yan

Zhenfeng

et al. 2020

IEEE Trans. Ind. Electron.

View full text Add to dashboard Cite

show abstract

“…A method for estimating the location of pointing gesture is also introduced, in which a NAO Robot has to approach a specific location and analyze which objects are present, trying to deduce which object the user was referring to. Besides vision-based gesture interface, inertial sensors are also widely investigated in Stančić et al or Jiang et al [36,37].…”

Section: Hri In Service Roboticsmentioning

confidence: 99%

Review of Interfaces for Industrial Human-Robot Interaction

Berg

2020

Curr Robot Rep

View full text Add to dashboard Cite

Purpose of Review The review presents an overview of research approaches on human-robot interfaces in industrial and service robotics. Recent Findings Research approaches address especially speech and gesture recognition in both fields but are more explored in service robotics. However, the importance of interfaces in industrial robots increases. Summary The development of human-robot interfaces is leading towards intuitive interfaces, especially using speech and gesture recognition and combining them to multimodal interfaces.

show abstract

“…Finger movements are linked with the physical operation of tendons, bones, and ligaments at the wrist [4][5][6][7]. Some studies have shown that an optical sensor [4], accelerometer and gyroscope [5,6], an array of barometric sensors [8], and a combination of sEMG, accelerometer, and gyroscope [9] around the wrist could be used for hand gesture recognition at the wrist. An inertial measurement unit (IMU), containing an accelerometer and a gyroscope, is a lightweight tiny chip that can be easily placed over the wrist [6].…”

Section: Introductionmentioning

confidence: 99%

“…An inertial measurement unit (IMU), containing an accelerometer and a gyroscope, is a lightweight tiny chip that can be easily placed over the wrist [6]. However, many studies have shown that an IMU sensor placed over the wrist in skin contact could be insufficient as a sole modality for hand gesture recognition, especially during arm movements [9]. If the IMU is not placed in skin contact, it may not detect information that could be recorded from the vibration over the skin while making a gesture.…”

Section: Introductionmentioning

confidence: 99%

“…If the IMU is not placed in skin contact, it may not detect information that could be recorded from the vibration over the skin while making a gesture. Many researchers have used IMU with other modalities to increase the accuracy of gesture recognition, e.g., IMU with sEMG [9]. In this study, we aimed to use pressure-based sensing with IMU at the wrist using pressurebased sensing units.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Multimodal hand gesture recognition using single IMU and acoustic measurements at wrist

Siddiqui

Chan

2020

PLoS ONE

View full text Add to dashboard Cite

To facilitate hand gesture recognition, we investigated the use of acoustic signals with an accelerometer and gyroscope at the human wrist. As a proof-of-concept, the prototype consisted of 10 microphone units in contact with the skin placed around the wrist along with an inertial measurement unit (IMU). The gesture recognition performance was evaluated through the identification of 13 gestures used in daily life. The optimal area for acoustic sensor placement at the wrist was examined using the minimum redundancy and maximum relevance feature selection algorithm. We recruited 10 subjects to perform over 10 trials for each set of hand gestures. The accuracy was 75% for a general model with the top 25 features selected, and the intra-subject average classification accuracy was over 80% with the same features using one microphone unit at the mid-anterior wrist and an IMU. These results indicate that acoustic signatures from the human wrist can aid IMU sensing for hand gesture recognition, and the selection of a few common features for all subjects could help with building a general model. The proposed multimodal framework helps address the single IMU sensing bottleneck for hand gestures during arm movement and/or locomotion. OPEN ACCESSCitation: Siddiqui N, Chan RHM (2020) Multimodal hand gesture recognition using single IMU and acoustic measurements at wrist. PLoS ONE 15(1): e0227039. https://doi.org/10.

show abstract

Feasibility of Wrist-Worn, Real-Time Hand, and Surface Gesture Recognition via sEMG and IMU Sensing

Cited by 181 publications

References 33 publications

A Proportional Pattern Recognition Control Scheme for Wearable A-mode Ultrasound Sensing

A Proportional Pattern Recognition Control Scheme for Wearable A-mode Ultrasound Sensing

Review of Interfaces for Industrial Human-Robot Interaction

Multimodal hand gesture recognition using single IMU and acoustic measurements at wrist

Contact Info

Product

Resources

About