Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications

Abstract

Deep neural networks (DNNs) have achieved unprecedented success in the field of artificial intelligence (AI), including computer vision, natural language processing, and speech recognition. However, their superior performance comes at the considerable cost of computational complexity, which greatly hinders their applications in many resource-constrained devices, such as mobile phones and Internet of Things (IoT) devices. Therefore, methods and techniques that are able to lift the efficiency bottleneck while preserving the high accuracy of DNNs are in great demand to enable numerous edge AI applications. This article provides an overview of efficient deep learning methods, systems, and applications. We start from introducing popular model compression methods, including pruning, factorization, quantization, as well as compact model design. To reduce the large design cost of these manual solutions, we discuss the AutoML framework for each of them, such as neural architecture search (NAS) and automated pruning and quantization. We then cover efficient on-device training to enable user customization based on the local data on mobile devices. Apart from general acceleration techniques, we also showcase several task-specific accelerations for point cloud, video, and natural language processing by exploiting their spatial sparsity and temporal/token redundancy. Finally, to support all these algorithmic advancements, we introduce the efficient deep learning system design from both software and hardware perspectives.

Deep Neural Network-Based Video Processing to Obtain Dual-Task Upper-Extremity Motor Performance Toward Assessment of Cognitive and Motor Function

“…In this way, we can construct a specialized video dataset focusing on upper-extremity movements, and further develop a more accurate deep neural network model for assessing cognitive and motor function by automatically extracting key parameters of upper-extremity motion signals from the video and directly predicting MMSE and TUG scores. Additionally, the model could be more lightweight and computational efficient to be deployed in a smartphone platform without need to upload the video recordings to the cloud for post-processing [ 38 ]. The video processing model running in local computation resource of the smartphone can also avoid disclosure of users’ facial information and identification, thereby addressing their privacy concerns.…”

Substation meter detection and recognition method based on lightweight deep learning model

“…Therefore, methods and techniques that can remove the efficiency bottleneck while maintaining the high accuracy of deep neural networks are highly needed to enable numerous edge applications. The current lightweight implementation of deep neural network models can be divided into two categories 17 , namely compact network design and model compression technology. The compact network design method does not compress the pre-trained network model, but directly designs a new network with smaller computational complexity and parameter amount, such as MobileNet .…”

Trends and prediction in daily incidence of novel coronavirus infection in China, Hubei Province and Wuhan City: an application of Farr’s law

“…The three widely used convolution operations are 1x1 convolution, group convolution, and depthwise convolution(Fig. 21) [108]. This section will focus on these three techniques that, when combined with skip connections, facilitate efficient learning.…”