Design and Implementation of a Convolutional Neural Network on an Edge Computing Smartphone for Human Activity Recognition

Abstract

Edge computing aims to integrate computing into everyday settings, enabling the system to be context-aware and private to the user. With the increasing success and popularity of deep learning methods, there is an increased demand to leverage these techniques in mobile and wearable computing scenarios. In this paper, we present an assessment of a deep human activity recognition system's memory and execution time requirements, when implemented on a mid-range smartphone class hardware and the memory implications for embedded hardware. This paper presents the design of a convolutional neural network (CNN) in the context of human activity recognition scenario. Here, layers of CNN automate the feature learning and the influence of various hyper-parameters such as the number of filters and filter size on the performance of CNN. The proposed CNN showed increased robustness with better capability of detecting activities with temporal dependence compared to models using statistical machine learning techniques. The model obtained an accuracy of 96.4% in a five-class static and dynamic activity recognition scenario. We calculated the proposed model memory consumption and execution time requirements needed for using it on a mid-range smartphone. Per-channel quantization of weights and per-layer quantization of activation to 8-bits of precision post-training produces classification accuracy within 2% of floating-point networks for dense, convolutional neural network architecture. Almost all the size and execution time reduction in the optimized model was achieved due to weight quantization. We achieved more than four times reduction in model size when optimized to 8-bit, which ensured a feasible model capable of fast on-device inference. INDEX TERMS Convolutional neural networks, edge computing, tensorflow lite, activity recognition, deep learning.

COVID-19 detection and disease progression visualization: Deep learning on chest X-rays for classification and coarse localization

“…Hence, another future direction from this research will be extending the model implementation on conventional smartphone processor to do fast and cheap on-device inference [14]. To provide a proof of concept of transferring the capability of deep learning models on mobile devices, we would like to build on our previous experience in transferring such models using the TensorFlow lite (TFlite) library [1,24]. Professor, Heriot-Watt University, UK for his kind guidance and comments that greatly improved the manuscript.…”

COVID-19 detection and disease progression visualization: Deep learning on chest X-rays for classification and coarse localization

“…Hence, another future direction from this research will be to extend the trained models implementation on conventional smartphone processor to do fast and cheap on-device inference to provide a proof of concept of transferring the capability of deep learning models on mobile devices [12]. We would like to build on our previous experience in transferring such models using the TensorFlow lite (TFlite) library [1,21].…”

Progress in Characterizing the Human Exposome: a Key Step for Precision Medicine

“…The wearable clinic app for schizophrenia integrates short smartphone questionnaires for psychosis symptom assessment [27][28][29], GPS sensing for behavioural phenotyping [30][31][32][33], and real-time risk assessment using cluster Hidden Markov models [34]. The wearable clinic for ABPM is an example of multimodal wearable sensors integration and uses activity classification from accelerometer data [35,36] to determine the optimal moments for blood pressure measurement [37].…”