Exploring Automatic Gym Workouts Recognition Locally on Wearable Resource-Constrained Devices

Abstract

Automatic gym activity recognition on energyand resource-constrained wearable devices removes the humaninteraction requirement during intense gym sessions -like softtouch tapping and swiping. This work presents a tiny and highly accurate residual convolutional neural network that runs in milliwatt microcontrollers for automatic workouts classification. We evaluated the inference performance of the deep model with quantization on three resource-constrained devices: two microcontrollers with ARM-Cortex M4 and M7 core from ST Microelectronics, and a GAP8 system on chip, which is an opensourced, multi-core RISC-V computing platform from Green-Waves Technologies. Experimental results show an accuracy of up to 90.4% for eleven workouts recognition with full precision inference. The paper also presents the trade-off performance of the resource-constrained system. While keeping the recognition accuracy (88.1%) with minimal loss, each inference takes only 3.2 ms on GAP8, benefiting from the 8 RISC-V cluster cores. We measured that it features an execution time that is 18.9x and 6.5x faster than the Cortex-M4 and Cortex-M7 cores, showing the feasibility of real-time on-board workouts recognition based on the described data set with 20 Hz sampling rate. The energy consumed for each inference on GAP8 is 0.41 mJ compared to 5.17 mJ on Cortex-M4 and 8.07 mJ on Cortex-M7 with the maximum clock. It can lead to longer battery life when the system is battery-operated. We also introduced an open data set composed of fifty sessions of eleven gym workouts collected from ten subjects that is publicly available.

Deep learning and transfer learning for device-free human activity recognition: A survey

“…Fig. 1(a) shows the employed model, which is a residual network consisting of 10 1D-convolutional layers and 1 fully connected (dense) layer [37]. The whole network can be divided into two parts: the backbone and the classifier.…”

“…A detailed description of the data set can be found in [66], in which the authors deployed a Resnet neural network for workout recognition and achieved an accuracy of 91%. The authors also tried a real-time edge evaluation with the compressed network in [37] and reported an accuracy of 88%.…”

“…In each round of evaluation, the network introduced in III-A is trained with Adam optimizer with 0.001 learning rate, 0.9 β 1 and 0.999 β 2 , combined with an early-stopping regularization with a patience value of 100. Following the method introduced in [37], samples in the RecGym dataset are assigned with weighted importance due to the imbalanced label distribution. While in QVAR and Ultra datasets, all samples are considered to have equal importance during offline training.…”

“…Other examples are Hailo-8 [33] from Hailo AI, STM32N6 [34] from ST Semiconductor, Spresense from Sony, and many others. Among them, the GAP series from Greenwaves Technology, powered mainly by a cluster of low-power RISC-V cores, have been proven to be among the first-class for edge intelligent applications [35]- [37]. The latest product, GAP9, utilizes a multi-core RISC-V cluster combined with a neural network acceleration called NE16, achieving a perfect balance between ultra-low power consumption, latency, flexibility, and ease of use.…”