Deep Learning-based Pose Estimation in Providing Feedback for Physical Movement: a Review

Abstract

Pose estimation has various applications in analyzing human movement and behavior, including providing feedback to users about their movements so they could adjust and improve their movement skills. To investigate the current research status and possible gaps, we searched Scopus and Web of Science for articles that (1) human `body' pose estimation is used and (2) user movement is assessed and communicated. We used either a bottom-up or top-down approach to analyze 20 articles for methods used to estimate human pose, assess movement, provide feedback to users, as well as methods to evaluate them. Our review found that pose estimation systems typically used CNNs while movement assessment methods varied from mathematical formulas or models, rule-based approaches, to machine learning. Feedback was primarily presented visually in verbal forms and nonverbal forms. The experiments to evaluate each part ranged from the use of public datasets to human participants. We found that while there was an improvement, the majority of pose estimation challenges remain. The effectiveness and factors for choosing movement assessment methods for a new context are still unclear. In the end, we suggest that studies about feedback prioritization and erroneous feedback are needed.

Design recommendations for XR-based motor rehabilitation exergames at home

“…The first two suggestions, however, need external intervention to choose the correct exercises, which can be impractical for home training. Machine learning-based approaches could assess the movement during training and adapt specific game parameters and feedback to adjust it to the patient's changing needs automatically, as suggested by SME and literature (Osgouei et al, 2020;Tharatipyakul and Pongnumkul, 2023). Although this technology has its first applications in fitness training, its feasibility in such a complex field as rehabilitation is yet to be systematically assessed.…”