Encoding human actions with a frequency domain approach

Shah, Drishti; Falco, Pietro; Saveriano, Matteo; Lee, Dongheui

doi:10.1109/iros.2016.7759780

Cited by 7 publications

(4 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Besides action recognition using video or using joints coordinates data, action recognition can be achieved by relying on motion data in the frequency domain. Several studies have converted human motion to the frequency domain using different methods [ 178 , 179 ] and used this additional information in the frequency domain for action recognition [ 180 , 181 ] or even for autoencoder-based motion generation [ 182 ]. Action recognition using information in the frequency domain also allows for faster performances, as compressed videos would be sufficient instead of regular RGB videos [ 181 ].…”

Section: Machine Learning Algorithms For Human Motion Analysismentioning

confidence: 99%

Using Artificial Intelligence for Assistance Systems to Bring Motor Learning Principles into Real World Motor Tasks

Vandevoorde

Vollenkemper

Schwan

et al. 2022

Sensors

View full text Add to dashboard Cite

Humans learn movements naturally, but it takes a lot of time and training to achieve expert performance in motor skills. In this review, we show how modern technologies can support people in learning new motor skills. First, we introduce important concepts in motor control, motor learning and motor skill learning. We also give an overview about the rapid expansion of machine learning algorithms and sensor technologies for human motion analysis. The integration between motor learning principles, machine learning algorithms and recent sensor technologies has the potential to develop AI-guided assistance systems for motor skill training. We give our perspective on this integration of different fields to transition from motor learning research in laboratory settings to real world environments and real world motor tasks and propose a stepwise approach to facilitate this transition.

show abstract

Section: Machine Learning Algorithms For Human Motion Analysismentioning

confidence: 99%

Using Artificial Intelligence for Assistance Systems to Bring Motor Learning Principles into Real World Motor Tasks

Vandevoorde

Vollenkemper

Schwan

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…In [21], an improved algorithm based on Fast Fourier Transform (FFT) is proposed which extracts features from resultant acceleration of the data obtained from smartphones. In [22], a frequency-based action descriptor called "FADE" is proposed to represent human actions. FFT is used to transform the signals into frequency domain and then these signals are resampled to exploit the frequency domain features.…”

Section: Related Workmentioning

confidence: 99%

Inertial Sensor Data to Image Encoding for Human Action Recognition

Ahmad

Khan

2021

IEEE Sensors J.

View full text Add to dashboard Cite

Convolutional Neural Networks (CNNs) are successful deep learning models in the field of computer vision. To get the maximum advantage of CNN model for Human Action Recognition (HAR) using inertial sensor data, in this paper, we use four types of spatial domain methods for transforming inertial sensor data to activity images, which are then utilized in a novel fusion framework. These four types of activity images are Signal Images (SI), Gramian Angular Field (GAF) Images, Markov Transition Field (MTF) Images and Recurrence Plot (RP) Images. Furthermore, for creating a multimodal fusion framework and to exploit activity images, we made each type of activity images multimodal by convolving with two spatial domain filters : Prewitt filter and High-boost filter. ResNet-18, a CNN model, is used to learn deep features from multi-modalities. Learned features are extracted from the last pooling layer of each ResNet and then fused by canonical correlation based fusion (CCF) for improving the accuracy of human action recognition. These highly informative features are served as input to a multiclass Support Vector Machine (SVM). Experimental results on three publicly available inertial datasets show the superiority of the proposed method over the current state-of-the-art.

show abstract

“…Diffusion score is used to determine best fractional order. In [13], a frequency-based action descriptor called FADE is proposed to represent human actions. FFT is used to transform the signals into frequency domain and then these signals are resampled to exploit the frequency domain features.…”

Section: Related Workmentioning

confidence: 99%

Multidomain Multimodal Fusion for Human Action Recognition Using Inertial Sensors

Ahmad

Khan

2019

2019 IEEE Fifth International Conference on Multimedia Big Data (BigMM)

View full text Add to dashboard Cite

One of the major reasons for misclassification of multiplex actions during action recognition is the unavailability of complementary features that provide the semantic information about the actions. In different domains these features are present with different scales and intensities. In existing literature, features are extracted independently in different domains but the benefits from fusing these multidomain features are not realized. To address this challenge and to extract complete set of complementary information, in this paper, we propose a novel multidomain multimodal fusion framework that extracts complementary and distinct features from different domains of the input modality. We transform input inertial data into signal images, and then made the input modality multidomain and multimodal by transforming spatial domain information into frequency and time-spectrum domain using Discrete Fourier Transform (DFT) and Gabor wavelet transform (GWT) respectively. Features in different domains are extracted by Convolutional Neural networks (CNNs) and then fused by canonical correlation based fusion (CCF) for improving the accuracy of human action recognition. Experimental results on three inertial datasets show the superiority of the proposed method when compared to the state-of-the-art.

show abstract

Encoding human actions with a frequency domain approach

Cited by 7 publications

References 27 publications

Using Artificial Intelligence for Assistance Systems to Bring Motor Learning Principles into Real World Motor Tasks

Using Artificial Intelligence for Assistance Systems to Bring Motor Learning Principles into Real World Motor Tasks

Inertial Sensor Data to Image Encoding for Human Action Recognition

Multidomain Multimodal Fusion for Human Action Recognition Using Inertial Sensors

Contact Info

Product

Resources

About