Real-Time Human Activity Recognition with IMU and Encoder Sensors in Wearable Exoskeleton Robot via Deep Learning Networks

Jaramillo, Ismael Espinoza; Jeong, Jin Gyun; Lopez, Patricio Rivera; Lee, Choong-Ho; Kang, Do-Yeon; Ha, Tae‐Jun; Oh, Jiheon; Jung, Hwanseok; Lee, Jin Hyuk; Lee, Won Hee; Kim, Tae-Seong

doi:10.3390/s22249690

Cited by 24 publications

(19 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Additionally, as the model complexity amplified, the SoftMax loss also escalated, suggesting that the concurrent use of CNN and LSTM layers did not enhance the outcomes. Jaramillo et al (2022) utilized a technique called Quaternion filtration by using single sensor data. In the next step, different segmentation techniques have been used to segment the data.…”

Section: Literature Reviewmentioning

confidence: 99%

Robust human locomotion and localization activity recognition over multisensory

Khan,

Alonazi,

Abdelhaq

et al. 2024

Front. Physiol.

View full text Add to dashboard Cite

Human activity recognition (HAR) plays a pivotal role in various domains, including healthcare, sports, robotics, and security. With the growing popularity of wearable devices, particularly Inertial Measurement Units (IMUs) and Ambient sensors, researchers and engineers have sought to take advantage of these advances to accurately and efficiently detect and classify human activities. This research paper presents an advanced methodology for human activity and localization recognition, utilizing smartphone IMU, Ambient, GPS, and Audio sensor data from two public benchmark datasets: the Opportunity dataset and the Extrasensory dataset. The Opportunity dataset was collected from 12 subjects participating in a range of daily activities, and it captures data from various body-worn and object-associated sensors. The Extrasensory dataset features data from 60 participants, including thousands of data samples from smartphone and smartwatch sensors, labeled with a wide array of human activities. Our study incorporates novel feature extraction techniques for signal, GPS, and audio sensor data. Specifically, for localization, GPS, audio, and IMU sensors are utilized, while IMU and Ambient sensors are employed for locomotion activity recognition. To achieve accurate activity classification, state-of-the-art deep learning techniques, such as convolutional neural networks (CNN) and long short-term memory (LSTM), have been explored. For indoor/outdoor activities, CNNs are applied, while LSTMs are utilized for locomotion activity recognition. The proposed system has been evaluated using the k-fold cross-validation method, achieving accuracy rates of 97% and 89% for locomotion activity over the Opportunity and Extrasensory datasets, respectively, and 96% for indoor/outdoor activity over the Extrasensory dataset. These results highlight the efficiency of our methodology in accurately detecting various human activities, showing its potential for real-world applications. Moreover, the research paper introduces a hybrid system that combines machine learning and deep learning features, enhancing activity recognition performance by leveraging the strengths of both approaches.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Robust human locomotion and localization activity recognition over multisensory

Khan,

Alonazi,

Abdelhaq

et al. 2024

Front. Physiol.

View full text Add to dashboard Cite

show abstract

“…Segmentation is an important concept used in signal processing. The concept of windowing and segmentation [50][51][52][53][54] involves dividing signals into smaller windows instead of processing complete or long sequences. The advantage of windowing is that it allows for easier data processing, reducing complexity and processing time.…”

Section: Signal Windowing and Segmentationmentioning

confidence: 99%

Biosensor-Driven IoT Wearables for Accurate Body Motion Tracking and Localization

Almujally,

Khan,

Al Mudawi

et al. 2024

Sensors

View full text Add to dashboard Cite

The domain of human locomotion identification through smartphone sensors is witnessing rapid expansion within the realm of research. This domain boasts significant potential across various sectors, including healthcare, sports, security systems, home automation, and real-time location tracking. Despite the considerable volume of existing research, the greater portion of it has primarily concentrated on locomotion activities. Comparatively less emphasis has been placed on the recognition of human localization patterns. In the current study, we introduce a system by facilitating the recognition of both human physical and location-based patterns. This system utilizes the capabilities of smartphone sensors to achieve its objectives. Our goal is to develop a system that can accurately identify different human physical and localization activities, such as walking, running, jumping, indoor, and outdoor activities. To achieve this, we perform preprocessing on the raw sensor data using a Butterworth filter for inertial sensors and a Median Filter for Global Positioning System (GPS) and then applying Hamming windowing techniques to segment the filtered data. We then extract features from the raw inertial and GPS sensors and select relevant features using the variance threshold feature selection method. The extrasensory dataset exhibits an imbalanced number of samples for certain activities. To address this issue, the permutation-based data augmentation technique is employed. The augmented features are optimized using the Yeo–Johnson power transformation algorithm before being sent to a multi-layer perceptron for classification. We evaluate our system using the K-fold cross-validation technique. The datasets used in this study are the Extrasensory and Sussex Huawei Locomotion (SHL), which contain both physical and localization activities. Our experiments demonstrate that our system achieves high accuracy with 96% and 94% over Extrasensory and SHL in physical activities and 94% and 91% over Extrasensory and SHL in the location-based activities, outperforming previous state-of-the-art methods in recognizing both types of activities.

show abstract

“…These sensors, affixed to the body’s joints, capture three-dimensional linear accelerations and angular velocities, facilitating the computation of full-body motion in a rapid (>200 Hz) and reliable manner. Prior research has explored the application of various machine learning techniques to process raw IMU data for action classification, including recurrent neural networks (RNNs) [ 35 ], LSTM networks [ 36 ], CNNs [ 37 ], and hybrid CNN-LSTM models [ 38 ], showcasing the potential of IMU sensors in overcoming the challenges posed by vision-based action recognition methods.…”

Section: Introductionmentioning

confidence: 99%

Action Recognition of Taekwondo Unit Actions Using Action Images Constructed with Time-Warped Motion Profiles

Lim,

Luo,

Lee

et al. 2024

Sensors

View full text Add to dashboard Cite

Taekwondo has evolved from a traditional martial art into an official Olympic sport. This study introduces a novel action recognition model tailored for Taekwondo unit actions, utilizing joint-motion data acquired via wearable inertial measurement unit (IMU) sensors. The utilization of IMU sensor-measured motion data facilitates the capture of the intricate and rapid movements characteristic of Taekwondo techniques. The model, underpinned by a conventional convolutional neural network (CNN)-based image classification framework, synthesizes action images to represent individual Taekwondo unit actions. These action images are generated by mapping joint-motion profiles onto the RGB color space, thus encapsulating the motion dynamics of a single unit action within a solitary image. To further refine the representation of rapid movements within these images, a time-warping technique was applied, adjusting motion profiles in relation to the velocity of the action. The effectiveness of the proposed model was assessed using a dataset compiled from 40 Taekwondo experts, yielding remarkable outcomes: an accuracy of 0.998, a precision of 0.983, a recall of 0.982, and an F1 score of 0.982. These results underscore this time-warping technique’s contribution to enhancing feature representation, as well as the proposed method’s scalability and effectiveness in recognizing Taekwondo unit actions.

show abstract

Real-Time Human Activity Recognition with IMU and Encoder Sensors in Wearable Exoskeleton Robot via Deep Learning Networks

Cited by 24 publications

References 30 publications

Robust human locomotion and localization activity recognition over multisensory

Robust human locomotion and localization activity recognition over multisensory

Biosensor-Driven IoT Wearables for Accurate Body Motion Tracking and Localization

Action Recognition of Taekwondo Unit Actions Using Action Images Constructed with Time-Warped Motion Profiles

Contact Info

Product

Resources

About