Application of Skeleton Data and Long Short-Term Memory in Action Recognition of Children with Autism Spectrum Disorder

Zhang, Yunkai; Tian, Yinghong; Wu, Pingyi; Chen, Dongfan

doi:10.3390/s21020411

Cited by 18 publications

(10 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Additionally, skeleton normality has been assessed by measuring angles and velocities with these methods [ 35 , 36 , 37 ]. These techniques are beneficial not only for generating three-dimensional poses [ 38 , 39 ] but also for understanding the correlation between postural behavior and functional disorders such as Parkinson’s disease [ 40 , 41 ], autism spectrum disorder [ 42 ], and metatarsophalangeal joint flexions [ 43 ]. OpenPose-based deep learning approaches are also instrumental in detecting skeletal, ankle, and foot motion [ 44 ], evaluating physical function [ 45 , 46 ], and studying post-stroke conditions [ 47 ].…”

Section: Discussionmentioning

confidence: 99%

Integrating OpenPose and SVM for Quantitative Postural Analysis in Young Adults: A Temporal-Spatial Approach

Lee,

Chen,

Lin

et al. 2024

Bioengineering

View full text Add to dashboard Cite

Noninvasive tracking devices are widely used to monitor real-time posture. Yet significant potential exists to enhance postural control quantification through walking videos. This study advances computational science by integrating OpenPose with a Support Vector Machine (SVM) to perform highly accurate and robust postural analysis, marking a substantial improvement over traditional methods which often rely on invasive sensors. Utilizing OpenPose-based deep learning, we generated Dynamic Joint Nodes Plots (DJNP) and iso-block postural identity images for 35 young adults in controlled walking experiments. Through Temporal and Spatial Regression (TSR) models, key features were extracted for SVM classification, enabling the distinction between various walking behaviors. This approach resulted in an overall accuracy of 0.990 and a Kappa index of 0.985. Cutting points for the ratio of top angles (TAR) and the ratio of bottom angles (BAR) effectively differentiated between left and right skews with AUC values of 0.772 and 0.775, respectively. These results demonstrate the efficacy of integrating OpenPose with SVM, providing more precise, real-time analysis without invasive sensors. Future work will focus on expanding this method to a broader demographic, including individuals with gait abnormalities, to validate its effectiveness across diverse clinical conditions. Furthermore, we plan to explore the integration of alternative machine learning models, such as deep neural networks, enhancing the system’s robustness and adaptability for complex dynamic environments. This research opens new avenues for clinical applications, particularly in rehabilitation and sports science, promising to revolutionize noninvasive postural analysis.

show abstract

Section: Discussionmentioning

confidence: 99%

Integrating OpenPose and SVM for Quantitative Postural Analysis in Young Adults: A Temporal-Spatial Approach

Lee,

Chen,

Lin

et al. 2024

Bioengineering

View full text Add to dashboard Cite

show abstract

“…A wealth of children's motion parameters can be observed in ASD study using a non-invasive method 24,25,32 . OpenPose is a common and robust tool used in pose estimation of children in ordinary 2D videos compared to many existing approaches [33][34][35][36][37] , e.g., Kinect. In this study, we developed a non-invasive computer vision and machine learning-based framework for analyzing ADOS assessment videos.…”

Section: Identifying Activity Level Related Movement Features Of Chil...mentioning

confidence: 99%

Identifying activity level related movement features of children with ASD based on ADOS videos

Jin

Zhu

Cao

et al. 2023

Sci Rep

View full text Add to dashboard Cite

Autism spectrum disorder (ASD) is a neurodevelopmental disorder that affects about 2% of children. Due to the shortage of clinicians, there is an urgent demand for a convenient and effective tool based on regular videos to assess the symptom. Computer-aided technologies have become widely used in clinical diagnosis, simplifying the diagnosis process while saving time and standardizing the procedure. In this study, we proposed a computer vision-based motion trajectory detection approach assisted with machine learning techniques, facilitating an objective and effective way to extract participants’ movement features (MFs) to identify and evaluate children’s activity levels that correspond to clinicians’ professional ratings. The designed technique includes two key parts: (1) Extracting MFs of participants’ different body key points in various activities segmented from autism diagnostic observation schedule (ADOS) videos, and (2) Identifying the most relevant MFs through established correlations with existing data sets of participants’ activity level scores evaluated by clinicians. The research investigated two types of MFs, i.e., pixel distance (PD) and instantaneous pixel velocity (IPV), three participants’ body key points, i.e., neck, right wrist, and middle hip, and five activities, including Table-play, Birthday-party, Joint-attention, Balloon-play, and Bubble-play segmented from ADOS videos. Among different combinations, the high correlations with the activity level scores evaluated by the clinicians (greater than 0.6 with p < 0.001) were found in Table-play activity for both the PD-based MFs of all three studied key points and the IPV-based MFs of the right wrist key point. These MFs were identified as the most relevant ones that could be utilized as an auxiliary means for automating the evaluation of activity levels in the ASD assessment.

show abstract

“…This method can effectively reduce data noise and has high efficiency in pose matching. This has a positive promotion effect on the treatment of sick children [9]. Some scholars have integrated OpenPose with other methods.…”

Section: Related Workmentioning

confidence: 99%

Practical Analysis of Digital Technology in an Electronic Education Platform in PE Training

Luo

2023

Int. J. Emerg. Technol. Learn.

View full text Add to dashboard Cite

To improve the practical effect of digital technology (DT) in electronic education platform (ECP) in sports training, this study proposes using OpenPose as the main method of posture matching in sports teaching. MobileNet was introduced to replace the backbone network in OpenPose to improve the efficiency of the model. At the same time, self-attention mechanism (SAM) is introduced to improve the accuracy of the model. In the performance verification results, the proposed algorithm has a mean mean absolute error (MAE) of 0.793 and a root mean square error (RMSE) of 0.628. The accuracy of the proposed algorithm reached a maximum of 0.893 in the test set. The accuracy of the proposed algorithm exceeds 90% in the training set. In the practice of physical education (PE) and training, the absolute error between the actual value and the predicted value does not exceed 1.5 cm, while the relative error does not exceed 1.2%. The minimum absolute error and relative error are 0.01 cm and 0.2%, respectively. The proposed algorithm has high prediction effect and performance, and can be applied to PE training.

show abstract

Application of Skeleton Data and Long Short-Term Memory in Action Recognition of Children with Autism Spectrum Disorder

Cited by 18 publications

References 47 publications

Integrating OpenPose and SVM for Quantitative Postural Analysis in Young Adults: A Temporal-Spatial Approach

Integrating OpenPose and SVM for Quantitative Postural Analysis in Young Adults: A Temporal-Spatial Approach

Identifying activity level related movement features of children with ASD based on ADOS videos

Practical Analysis of Digital Technology in an Electronic Education Platform in PE Training

Contact Info

Product

Resources

About