Bioformers: Embedding Transformers for Ultra-Low Power sEMG-based Gesture Recognition

Burrello, Alessio; Morghet, Francesco Bianco; Scherer, Moritz; Benatti, Simone; Benini, Luca; Macii, Enrico; Poncino, Massimo; Pagliari, Daniele Jahier

doi:10.23919/date54114.2022.9774639

Cited by 18 publications

(13 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this work, which extends [ 23 ], we demonstrate that efficient transformers specifically tailored for edge devices are able to achieve very high energy efficiency, while maintaining state-of-the-art performance. We focus in particular on using these networks for the sEMG-based gesture recognition task.…”

Section: Introductionsupporting

confidence: 68%

“…These approaches typically consist of three parts: (i) an analog front end for bio-potential acquisition, (ii) a data preprocessing and feature extraction/selection step, and (iii) a final classification back end. They often use classic ML algorithms, such as support vector machines (SVMs), RFs, linear discriminant analysis (LDA), or artificial neural networks (ANNs) [ 8 , 9 , 10 , 11 , 13 ], or more recently, DL ones [ 4 , 23 , 24 , 25 , 26 , 27 ]. For example, authors in [ 28 , 29 ] have achieved over 90% accuracy in hand gesture classification using ANNs with five time-domain features (mean absolute value, mean absolute value slope, number of slope sign changes, number of zero crossings, and waveform length).…”

Section: Related Workmentioning

confidence: 99%

“…The main challenge in sEMG-based gesture recognition has therefore shifted from achieving high classification accuracy to managing the variability of the signal, which is affected by various factors, such as anatomical variability, posture, fatigue, perspiration, changes in the skin-to-electrode interface, user adaptation, and electrode repositioning over multiple days of use [ 7 , 9 , 23 , 24 , 31 ]. These factors significantly hinder generalization, limiting long-term use and the development of robust real-time recognition systems.…”

Section: Related Workmentioning

confidence: 99%

“…Tiny transformers were first applied to this task in our previous work [ 23 ], where we demonstrated that they can reach a state-of-the-art accuracy of 65.73% (64.7% with int8 data representation) with an energy consumption of just 0.143 mJ per classification, dramatically increasing the battery lifetime of the edge device on which the network is deployed compared to the CNN of [ 24 ]. In this work, we further extend this approach, by improving the accuracy of ∼4%, while keeping a constant energy consumption.…”

Section: Related Workmentioning

confidence: 99%

See 3 more Smart Citations

Reducing the Energy Consumption of sEMG-Based Gesture Recognition at the Edge Using Transformers and Dynamic Inference

Xie

Burrello

Daghero

et al. 2023

Sensors

Self Cite

View full text Add to dashboard Cite

Hand gesture recognition applications based on surface electromiographic (sEMG) signals can benefit from on-device execution to achieve faster and more predictable response times and higher energy efficiency. However, deploying state-of-the-art deep learning (DL) models for this task on memory-constrained and battery-operated edge devices, such as wearables, requires a careful optimization process, both at design time, with an appropriate tuning of the DL models’ architectures, and at execution time, where the execution of large and computationally complex models should be avoided unless strictly needed. In this work, we pursue both optimization targets, proposing a novel gesture recognition system that improves upon the state-of-the-art models both in terms of accuracy and efficiency. At the level of DL model architecture, we apply for the first time tiny transformer models (which we call bioformers) to sEMG-based gesture recognition. Through an extensive architecture exploration, we show that our most accurate bioformer achieves a higher classification accuracy on the popular Non-Invasive Adaptive hand Prosthetics Database 6 (Ninapro DB6) dataset compared to the state-of-the-art convolutional neural network (CNN) TEMPONet (+3.1%). When deployed on the RISC-V-based low-power system-on-chip (SoC) GAP8, bioformers that outperform TEMPONet in accuracy consume 7.8×–44.5× less energy per inference. At runtime, we propose a three-level dynamic inference approach that combines a shallow classifier, i.e., a random forest (RF) implementing a simple “rest detector” with two bioformers of different accuracy and complexity, which are sequentially applied to each new input, stopping the classification early for “easy” data. With this mechanism, we obtain a flexible inference system, capable of working in many different operating points in terms of accuracy and average energy consumption. On GAP8, we obtain a further 1.03×–1.35× energy reduction compared to static bioformers at iso-accuracy.

show abstract

Section: Introductionsupporting

confidence: 68%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Reducing the Energy Consumption of sEMG-Based Gesture Recognition at the Edge Using Transformers and Dynamic Inference

Xie

Burrello

Daghero

et al. 2023

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…Regarding biological signals, Krishna et al [18] proposed an automatic speech recognition model based on Transformer using as input statistical features extracted from EEG signals. Other recent works also employed Transformerbased models in classification tasks using as input EEG, for emotion recognition [19], and EMG signals, for hand gesture classification [20]. These recent advances open up a new range of application areas.…”

Section: Introductionmentioning

confidence: 99%

Electromyography Based Decoding of Dexterous, In-Hand Manipulation Motions With Temporal Multichannel Vision Transformers

Godoy

Dwivedi

Liarokapis

2022

IEEE Trans. Neural Syst. Rehabil. Eng.

View full text Add to dashboard Cite

Electromyography (EMG) signals have been used in designing muscle-machine interfaces (MuMIs) for various applications, ranging from entertainment (EMG controlled games) to human assistance and human augmentation (EMG controlled prostheses and exoskeletons). For this, classical machine learning methods such as Random Forest (RF) models have been used to decode EMG signals. However, these methods depend on several stages of signal pre-processing and extraction of handcrafted features so as to obtain the desired output. In this work, we propose EMG based frameworks for the decoding of object motions in the execution of dexterous, inhand manipulation tasks using raw EMG signals input and two novel deep learning (DL) techniques called Temporal Multi-Channel Transformers and Vision Transformers. The results obtained are compared, in terms of accuracy and speed of decoding the motion, with RF-based models and Convolutional Neural Networks as a benchmark. The models are trained for 11 subjects in a motion-object specific and motion-object generic way, using the 10-fold crossvalidation procedure. This study shows that the performance of MuMIs can be improved by employing DL-based models with raw myoelectric activations instead of developing DL or classic machine learning models with handcrafted features.

show abstract

TranSenseFusers: A temporal CNN-Transformer neural network family for explainable PPG-based stress detection

Kasnesis,

Chatzigeorgiou,

Feidakis

et al. 2025

Biomedical Signal Processing and Control

View full text Add to dashboard Cite

Bioformers: Embedding Transformers for Ultra-Low Power sEMG-based Gesture Recognition

Cited by 18 publications

References 19 publications

Reducing the Energy Consumption of sEMG-Based Gesture Recognition at the Edge Using Transformers and Dynamic Inference

Reducing the Energy Consumption of sEMG-Based Gesture Recognition at the Edge Using Transformers and Dynamic Inference

Electromyography Based Decoding of Dexterous, In-Hand Manipulation Motions With Temporal Multichannel Vision Transformers

TranSenseFusers: A temporal CNN-Transformer neural network family for explainable PPG-based stress detection

Contact Info

Product

Resources

About