Skeleton-based Human Keypoints Detection and Action Similarity Assessment for Fitness Assistance

Zhou, Jingzhe; Feng, Wei; Lei, Qujiang; Liu, Xianyong; Zhong, Qiubo; Wang, Yuhe; Jin, Jintao; Gui, Guangchao

doi:10.1109/icsip52628.2021.9689020

Cited by 8 publications

(20 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our method is cheaper and does not have to consider the power consumption and hygiene issues of wearable devices. Compared to the method of Jiangkun Zhou et al [ 12 ], accuracy improved the performance by 29.65%. Experimental results showed that our proposed method has better performance.…”

Section: Methodsmentioning

confidence: 89%

“…Body-node-based methods detect fitness movements by calculating body nodes, which can be performed using OpenPose, Mediapipe, Simple Baselines, etc. [ 10 , 11 , 12 , 13 ]. Using these methods, nodes of the body and fitness movements can be detected through changes in the coordinates of the nodes.…”

Section: Introductionmentioning

confidence: 99%

“…Using these methods, nodes of the body and fitness movements can be detected through changes in the coordinates of the nodes. In addition to detecting the speed of fitness movements [ 10 ], these methods can also classify the current fitness movement type [ 11 ] or the error between fitness movements and standard movements [ 12 , 13 ].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Fitness Movement Types and Completeness Detection Using a Transfer-Learning-Based Deep Neural Network

Chen

Shin

Hasan

et al. 2022

Sensors

View full text Add to dashboard Cite

Fitness is important in people’s lives. Good fitness habits can improve cardiopulmonary capacity, increase concentration, prevent obesity, and effectively reduce the risk of death. Home fitness does not require large equipment but uses dumbbells, yoga mats, and horizontal bars to complete fitness exercises and can effectively avoid contact with people, so it is deeply loved by people. People who work out at home use social media to obtain fitness knowledge, but learning ability is limited. Incomplete fitness is likely to lead to injury, and a cheap, timely, and accurate fitness detection system can reduce the risk of fitness injuries and can effectively improve people’s fitness awareness. In the past, many studies have engaged in the detection of fitness movements, among which the detection of fitness movements based on wearable devices, body nodes, and image deep learning has achieved better performance. However, a wearable device cannot detect a variety of fitness movements, may hinder the exercise of the fitness user, and has a high cost. Both body-node-based and image-deep-learning-based methods have lower costs, but each has some drawbacks. Therefore, this paper used a method based on deep transfer learning to establish a fitness database. After that, a deep neural network was trained to detect the type and completeness of fitness movements. We used Yolov4 and Mediapipe to instantly detect fitness movements and stored the 1D fitness signal of movement to build a database. Finally, MLP was used to classify the 1D signal waveform of fitness. In the performance of the classification of fitness movement types, the mAP was 99.71%, accuracy was 98.56%, precision was 97.9%, recall was 98.56%, and the F1-score was 98.23%, which is quite a high performance. In the performance of fitness movement completeness classification, accuracy was 92.84%, precision was 92.85, recall was 92.84%, and the F1-score was 92.83%. The average FPS in detection was 17.5. Experimental results show that our method achieves higher accuracy compared to other methods.

show abstract

Section: Methodsmentioning

confidence: 89%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Fitness Movement Types and Completeness Detection Using a Transfer-Learning-Based Deep Neural Network

Chen

Shin

Hasan

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…For example, OpenPose 38 or other pose-detection software could be used to assess the similarity between participants' and original dancer's body positions over time 29 , which would yield more sensitive and objective measure of performance. Further, pose detection software could be used to obtain an objective measure of complexity, such as entropy 10,39,40 from each movement sequence, as complexity is known to influence aesthetic judgements. This measure of movement complexity may be included as a regressor in analyses 9,10,13 .…”

Section: Dance Performance Predicts Cortical Responses To Known Movem...mentioning

confidence: 99%

Aesthetic evaluation of body movements shaped by embodiment and arts experience: Insights from behaviour and fNIRS

Casale,

Moffat,

Cross

2023

Preprint

View full text Add to dashboard Cite

Aesthetic appreciation of full-body movements is likely shaped by our cumulative bodily experiences, yet the literature has focused on expertise and familiarity. We ran two experiments exploring individual differences in embodiment and experience with the arts: In Study 1, we explored how participants’ (n=41) abilities to learn a choreography shaped their aesthetic perceptions while viewing learned vs. unknown movements, using functional near-infrared spectroscopy (fNIRS) to measure cortical activation over the Action Observation Network (i.e., inferior frontal gyrus [IFG], inferior parietal lobule, middle temporal gyrus [MTG]). Study 1 demonstrated that embodiment enhanced ratings of enjoyment, familiarity, and reproducibility of movements, and that individual differences in participants’ performance of the learned choreography were not associated with aesthetic ratings, but rather cortical activation in IFG and right MTG while viewing embodied (learned) choreography. In Study 2, we combined the behavioural data from Study 1 with additional data (total n=141), to examine the relationship between arts experience and aesthetic perceptions of movements robustly. Study 2 revealed that previous arts and sports experience correlated with aesthetic judgements of familiarity and reproducibility of movements. Our findings highlight the relevance of examining individual experiences to fill theoretical gaps in our understanding of action aesthetics.

show abstract

“…2019 Dance [26], Ski [27], Tai Chi [28], Exercise [29], Rehabilitation [30] 5 2020 Rehabilitation [2] 1 2021 Ballet [31], Yoga [32][33][34], Tai Chi [35], Kickboxing [36], Baseball [37], Exercise [38][39][40][41]…”

Section: Year Articles Countmentioning

confidence: 99%

Deep Learning-based Pose Estimation in Providing Feedback for Physical Movement: a Review

Tharatipyakul¹,

Pongnumkul²

2023

Preprint

View full text Add to dashboard Cite

Pose estimation has various applications in analyzing human movement and behavior, including providing feedback to users about their movements so they could adjust and improve their movement skills. To investigate the current research status and possible gaps, we searched Scopus and Web of Science for articles that (1) human `body' pose estimation is used and (2) user movement is assessed and communicated. We used either a bottom-up or top-down approach to analyze 20 articles for methods used to estimate human pose, assess movement, provide feedback to users, as well as methods to evaluate them. Our review found that pose estimation systems typically used CNNs while movement assessment methods varied from mathematical formulas or models, rule-based approaches, to machine learning. Feedback was primarily presented visually in verbal forms and nonverbal forms. The experiments to evaluate each part ranged from the use of public datasets to human participants. We found that while there was an improvement, the majority of pose estimation challenges remain. The effectiveness and factors for choosing movement assessment methods for a new context are still unclear. In the end, we suggest that studies about feedback prioritization and erroneous feedback are needed.

show abstract

Skeleton-based Human Keypoints Detection and Action Similarity Assessment for Fitness Assistance

Cited by 8 publications

References 18 publications

Fitness Movement Types and Completeness Detection Using a Transfer-Learning-Based Deep Neural Network

Fitness Movement Types and Completeness Detection Using a Transfer-Learning-Based Deep Neural Network

Aesthetic evaluation of body movements shaped by embodiment and arts experience: Insights from behaviour and fNIRS

Deep Learning-based Pose Estimation in Providing Feedback for Physical Movement: a Review

Contact Info

Product

Resources

About