Towards Zero Re-Training for Long-Term Hand Gesture Recognition via Ultrasound Sensing

Abstract

While myoelectric pattern recognition is a prevailing way for gesture recognition, the inherent nonstationarity of electromyography signals hinders its long-term application. This study aims to prove a hypothesis that morphological information of muscle contraction detected by ultrasound image is potentially suitable for long-term use. A set of ultrasound-based algorithms are proposed to realize robust hand gesture recognition over multiple days, with user training only at the first day. A markerless calibration algorithm is first presented to position the ultrasound probe during donning and doffing; an algorithm combining speeded-up robust features (SURF) and bag-of-features (BoF) model being immune to ultrasound probe shift and rotation is then introduced; a self-enhancing classification method is next adopted to update classification model automatically by incorporating useful knowledge from testing data; finally the performance of long-term hand gesture recognition with zero re-training is validated by a six-day experiment of six healthy subjects, whose outcomes strongly support the hypothesis with about 94% of gesture recognition accuracy for each testing day. This study confirms the feasibility of adoption of ultrasound sensing for long-term musculature related applications.

Recent Advances in Machine Learning Applied to Ultrasound Imaging

“…For the purposes mentioned above, most of the techniques investigated are based on three principal issues, i.e., detection [59][60][61][62][63][64][65], segmentation [66][67][68][69][70][71][72][73][74][75][76][77], and classification [63,65,[78][79][80][81][82][83][84][85][86][87][88][89][90][91][92][93][94].…”

Control Methods for Transradial Prostheses Based on Remnant Muscle Activity and Its Relationship with Proprioceptive Feedback

“…Multiple approaches have been used to compensate for the shift of the probe. An approach proposed by Yang et al [ 91 ] consisted of manual adjustment of the probe during the preparation stage at each donning of the probe. This calibration step required an image matching algorithm which was used to compare the current placement’s 2D scan with the initial scan (obtained during the gesture classifier training process).…”

Noncontact Human–Machine Interface Using Complementary Information Fusion Based on MEMS and Triboelectric Sensors

“…Current solutions for realizing noncontact wearable HMIs include optical, radar and ultrasound sensors that can remotely detect human finger motions. [ 43 , 44 , 45 , 46 , 47 , 48 ] However, these sensors suffer from the drawbacks of high power consumption as well as complex signal processing circuits and detecting algorithms. A potential candidate to address these issues in noncontact HMIs is the triboelectric sensors with the significant advantages of self‐powered capability and facile configuration.…”