Physiotherapy Exercise Classification with Single-Camera Pose Detection and Machine Learning

Abstract

Access to healthcare, including physiotherapy, is increasingly occurring through virtual formats. At-home adherence to physical therapy programs is often poor and few tools exist to objectively measure participation. The aim of this study was to develop and evaluate the potential for performing automatic, unsupervised video-based monitoring of at-home low-back and shoulder physiotherapy exercises using a mobile phone camera. Joint locations were extracted from the videos of healthy subjects performing low-back and shoulder physiotherapy exercises using an open source pose detection framework. A convolutional neural network was trained to classify physiotherapy exercises based on the segments of keypoint time series data. The model’s performance as a function of input keypoint combinations was studied in addition to its robustness to variation in the camera angle. The CNN model achieved optimal performance using a total of 12 pose estimation landmarks from the upper and lower body (low-back exercise classification: 0.995 ± 0.009; shoulder exercise classification: 0.963 ± 0.020). Training the CNN on a variety of angles was found to be effective in making the model robust to variations in video filming angle. This study demonstrates the feasibility of using a smartphone camera and a supervised machine learning model to effectively classify at-home physiotherapy participation and could provide a low-cost, scalable method for tracking adherence to physical therapy exercise programs in a variety of settings.

Towards Automating Personal Exercise Assessment and Guidance with Affordable Mobile Technology

“…Numerous research studies have employed ML models using BlazePose keypoint data to predict falls in real-time scenarios [29,30]. To our knowledge, the work of Arrowsmith et al is the only other approach utilizing BlazePose for classifying physiotherapy exercises [31]. However, there was no emphasis on evaluating exercise performance, and the ML models were not trained or tested on actual patients.…”

STPT: Spatio-Temporal Polychromatic Trajectory Based Elderly Exercise Evaluation System

“…The study [19]suggested a technique for identifying human activity that combined Dynamic time warping and Laban movement analysis (LMA). In this paper [20], the authors have used Convolutional Neural Networks (CNN) alongside Support Vector Machine (SVM) on the time series data for exercise performance evaluation.…”

Exercise quantification from single camera view markerless 3D pose estimation

“…VideoPose [ 12 ]/PoseLifter [ 18 ]) exists but is generally not directly suitable for rehabilitation/sporting applications. Very recently, BlazePose [ 11 ] was applied in physiotherapy exercise classification [ 19 ], but the predictive capacity for individual joint angles was not clear. Further a new two-camera approach combining 2D keypoint estimation from OpenPose to lift to 3D yielded good accuracy in assessing mean absolute error in joint angles in a variety of activities, including walking and squatting and jumping [ 20 ], although the accuracy of individual metrics of interest to physiotherapists was not presented, and a calibration step is needed to combine the output from the two cameras.…”