Movement Estimation Using Soft Sensors Based on Bi-LSTM and Two-Layer LSTM for Human Motion Capture

Abstract

The importance of estimating human movement has increased in the field of human motion capture. HTC VIVE is a popular device that provides a convenient way of capturing human motions using several sensors. Recently, the motion of only users’ hands has been captured, thereby greatly reducing the range of motion captured. This paper proposes a framework to estimate single-arm orientations using soft sensors mainly by combining a Bi-long short-term memory (Bi-LSTM) and two-layer LSTM. Positions of the two hands are measured using an HTC VIVE set, and the orientations of a single arm, including its corresponding upper arm and forearm, are estimated using the proposed framework based on the estimated positions of the two hands. Given that the proposed framework is meant for a single arm, if orientations of two arms are required to be estimated, the estimations are performed twice. To obtain the ground truth of the orientations of single-arm movements, two Myo gesture-control sensory armbands are employed on the single arm: one for the upper arm and the other for the forearm. The proposed framework analyzed the contextual features of consecutive sensory arm movements, which provides an efficient way to improve the accuracy of arm movement estimation. In comparison with the ground truth, the proposed method estimated the arm movements using a dynamic time warping distance, which was the average of 73.90% less than that of a conventional Bayesian framework. The distinct feature of our proposed framework is that the number of sensors attached to end-users is reduced. Additionally, with the use of our framework, the arm orientations can be estimated with any soft sensor, and good accuracy of the estimations can be ensured. Another contribution is the suggestion of the combination of the Bi-LSTM and two-layer LSTM.

Analysis of Influence of Segmentation, Features, and Classification in sEMG Processing: A Case Study of Recognition of Brazilian Sign Language Alphabet

“…The commercial Myo TM armband was chosen for data acquisition, and is presented in Figure 2 a. This device is widely used in gesture recognition research [ 46 , 47 ]. Myo TM has eight equidistant channels, with 200 samples/s, 8-bit resolution from ADC (Analog to Digital Converter), and wireless communication via Bluetooth [ 48 ].…”

Applying Deep Learning Techniques to Estimate Patterns of Musical Gesture

“…The gestures are part of the Myo Dataset (Côté-Allard et al, 2019) and NinaPro DB5 (Pizzolato et al, 2017). Guo and Sung (2020) captured human motion utilizing the HTC-VIVE virtual reality device in synchronization with the Myo armband. They harnessed the Bi-LSTM and two-layer LSTM architecture to recognize 15 different motor actions using the arms in a 3D video-game context.…”

RNN- and LSTM-Based Soft Sensors Transferability for an Industrial Process

“…SS implementation often requires the use of black-box nonlinear dynamical identification strategies, which uses data collected from the distributed control system [ 11 ] and stored in the historical database. To achieve this aim, machine learning (ML) techniques are mostly used, ranging from Support Vector Regression [ 12 ], Partial Least Square [ 13 ], and classical multilayer perceptrons [ 1 , 14 , 15 , 16 , 17 ] to more recent deep architectures, such as deep belief networks [ 9 , 18 , 19 , 20 ], long short-term memory networks (LSTMs) [ 21 , 22 ], and stacked autoencoders [ 23 , 24 , 25 , 26 ]. Bayesian approaches [ 27 ], Gaussian Processes Regression [ 28 ], Extreme Learning Machines [ 29 ], and adaptive methods, [ 30 , 31 , 32 ] are also used.…”