Recognition and Repetition Counting for Complex Physical Exercises with Deep Learning

Abstract

Activity recognition using off-the-shelf smartwatches is an important problem in humanactivity recognition. In this paper, we present an end-to-end deep learning approach, able to provideprobability distributions over activities from raw sensor data. We apply our methods to 10 complexfull-body exercises typical in CrossFit, and achieve a classification accuracy of 99.96%. We additionallyshow that the same neural network used for exercise recognition can also be used in repetitioncounting. To the best of our knowledge, our approach to repetition counting is novel and performswell, counting correctly within an error of 1 repetitions in 91% of the performed sets.

Recognition and Repetition Counting for Local Muscular Endurance Exercises in Exercise-Based Rehabilitation: A Comparative Study Using Artificial Intelligence Models

“…The second approach investigated for repetition counting was a deep CNN model, based on the AlexNet architecture ( CNN_Model ). We compare a single deep CNN model for the repetition counting task of all the exercises as opposed to the use of multiple CNN models as used in [ 27 ]. Figure 7 illustrates the pipeline used for the repetition counting task using the CNN_Model as a binary classifier along with an additional repetition counter block.…”

“…The application of ML methods to study data from human movements and activities to detect and understand these activities are referred to as human activity recognition (HAR). In recent years, many ML and deep learning-based models have been used along with wearable sensors in the assessment of human movement activities in many domains including: health [ 11 ], recreation activities [ 12 ], musculoskeletal injuries or diseases [ 13 ], day-to-day routine activities (e.g., walking, jogging, running, sitting, drinking, watching TV) [ 11 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 ], sporting movements [ 22 ] and exercises [ 23 , 24 , 25 , 26 , 27 ]. The ML models used for exercise recognition have predominantly used multiple wearable sensors [ 28 , 29 , 30 , 31 ], specifically in the areas of free weight exercise monitoring [ 32 ], the performance of lunge evaluation [ 24 ], limb movement rehabilitation [ 33 ], intensity recognition in strength training [ 34 ], exercise feedback [ 24 ], qualitative evaluation of human movements [ 28 ], gym activity monitoring [ 29 ], rehabilitation [ 23 , 25 , 33 , 35 ] and indoor-based exercises for strength training [ 36 ].…”

“…Studies [ 8 , 15 , 17 , 19 ] on the usage of wearable sensors, either phone-based or using inertial measurement units, have shown that CVD patients (67~68%) have an interest in single sensor-based cardiac rehabilitation [ 8 ]. Exercise-based applications, using single sensors, include recognizing day-to-day activities [ 26 , 37 , 38 , 39 , 40 , 41 ], and multiple complex exercises [ 23 , 26 , 27 ] or single exercises such as lunges [ 24 ] and squats [ 42 ], as well as repetition counting [ 27 , 43 , 44 ]. Therefore, in our research, we will use a single wrist-worn inertial sensor for exercise recognition and repetition counting.…”

“…To date, the vast majority of HAR studies detailed above have used traditional ML approaches such as decision trees, Naive Bayes, random forest, perceptron neural networks, k-nearest neighbor and support vector machines. There is, however, a growing interest in the potential use of deep learning methods in the field of activity recognition mainly using CNN [ 27 , 46 , 47 , 48 , 49 ] and recurrent models [ 47 , 50 ]. A small number of studies [ 46 , 47 , 49 , 51 ] have shown the significant advantage of using deep learning models in the general area of HAR.…”

“…A small number of studies [ 46 , 47 , 49 , 51 ] have shown the significant advantage of using deep learning models in the general area of HAR. However, very few studies [ 23 , 25 , 27 , 30 ] appear to have used deep learning models in exercise recognition and repetition counting, and where employed they use multiple CNN models for the repetition counting task. To the best of our knowledge, there are no works reported using a single deep CNN model for exercise recognition and for repetition counting.…”