Human Activity Recognition on Microcontrollers with Quantized and Adaptive Deep Neural Networks

Abstract

Human Activity Recognition (HAR) based on inertial data is an increasingly diffused task on embedded devices, from smartphones to ultra low-power sensors. Due to the high computational complexity of deep learning models, most embedded HAR systems are based on simple and not-so-accurate classic machine learning algorithms. This work bridges the gap between on-device HAR and deep learning, proposing a set of efficient one-dimensional Convolutional Neural Networks (CNNs) that can be deployed on general purpose microcontrollers (MCUs). Our CNNs are obtained combining hyper-parameters optimization with sub-byte and mixed-precision quantization, to find good trade-offs between classification results and memory occupation. Moreover, we also leverage adaptive inference as an orthogonal optimization to tune the inference complexity at runtime based on the processed input, hence producing a more flexible HAR system. With experiments on four datasets, and targeting an ultra-low-power RISC-V MCU, we show that (i) we are able to obtain a rich set of Pareto-optimal CNNs for HAR, spanning more than 1 order of magnitude in terms of memory, latency, and energy consumption; (ii) thanks to adaptive inference, we can derive >20 runtime operating modes starting from a single CNN, differing by up to 10% in classification scores and by more than 3× in inference complexity, with a limited memory overhead; (iii) on three of the four benchmarks, we outperform all previous deep learning methods, while reducing the memory occupation by more than 100×. The few methods that obtain better performance (both shallow and deep) are not compatible with MCU deployment; (iv) all our CNNs are compatible with real-time on-device HAR, achieving an inference latency that ranges between 9 μs and 16 ms. Their memory occupation varies in 0.05–23.17 kB, and their energy consumption in 0.05 and 61.59 μJ, allowing years of continuous operation on a small battery supply.

Dynamic Decision Tree Ensembles for Energy-Efficient Inference on IoT Edge Nodes

“…Dynamic (or adaptive) inference techniques, including this work, are designed to overcome these limitations. They allow the deployment of a single model able to adapt its complexity at runtime, while keeping the memory overhead under control [24]- [26]. In practice, a dynamic model can be partially turned off when the external conditions require it, or when the processing input's difficulty allows it [22].…”

“…Literature works differ mainly in how they decompose the model. For instance, the authors of [24]- [26], [28] obtain a single sub-model by selectively deactivating a subset of the layers or channels of a network, or truncating the bit-width used to represent parameters. Other works extend the approach more than two sub-models [25], [29] or enhance the stopping criterion with class-aware thresholds [22].…”

AUTO-HAR: An adaptive human activity recognition framework using an automated CNN architecture design

“… The encoding strategy used for the devised CNN architecture has reduced the search space of the architecture, which prevents the structure from expressing the diversity of the assigned tasks. [15] Human Activity Recognition A simple grid search algorithm has been explored how to optimize the models' hyper-parameters. To achieve a good trade-off between classification score and memory occupancy, a combination of sub-byte and mixed precision quantization was used.…”

“…In addition, practitioners of deep learning encounter difficulty when it comes to manually building deep models and determining suitable configurations (e.g., model layers and operation types) through trial and error. Various steps are involved in feeding domain knowledge into DL, including Feature Engineering (FE) [13] , model generation [14] , and model deployment [15] , [16] . Because CNNs are based on layers, they allow the flexibility of adding or removing layers based on the training phase, which is then used in inference (classification) to classify the data.…”

Energy Efficiency of Deep Learning Compression Techniques in Wearable Human Activity Recognition

“…Daghero et al applied Binary Neural Networks (BNNs) to HAR to decrease network complexity via an extreme form of quantization [6]; indeed, by using BNNs the precision of data format, both weights and layers input/output, is reduced to 1-bit precision. Authors propose a BNN inference library that targets RISC-V processors.Subsequently, authors extended their work [4,5] by proposing a set of efficient one-dimensional convolutional neural networks (1D CNNs) and testing optimization techniques such as sub-type and mixed-precision quantization. The aim was to find a good trade-off between accuracy and memory occupation.…”