A collaborative CPU‐GPU approach for deep learning on mobile devices

Abstract

As mobile devices become more prevalent, users tend to reassess their expectations regarding the personalization of mobile services. The data collected by a mobile device's sensors provide an opportunity to gain insight into the user's profile. Recently, deep learning has gained momentum and has become the method of choice for solving machine learning problems. Interestingly, training a deep neural network on a mobile device is often mistakenly regarded as cumbersome.For instance, several deep learning frameworks only provide a CPU-based implementation for prediction tasks on a mobile device. In contrast to servers, a mobile computing environment imposes many domain-specific constraints that invite us to review the general computing approach used in a deep learning framework implementation. In this paper, we propose a deep learning framework that has been specifically designed for mobile device platforms. Our approach relies on the collaboration of the multicore CPU and the integrated GPU to accelerate deep learning computation on mobile devices. Our work exploits the shared memory architecture of mobile devices to promote CPU-GPU collaboration without any data copying. We analyze our approach with regard to three factors: performance/portability trade-off, power efficiency, and memory management. KEYWORDS deep learning, energy efficient, GPGPU, heterogeneous system, mobile computing, OpenCL INTRODUCTIONAs mobile devices get more sophisticated, the need for more optimization and personalization of mobile services has become a primary expectation of users. The latest mobile devices provide the opportunity to collect information about the user throughout the day via multiple mobile sensors, wearable sensors, and the Internet of things (IoT). In contrast, desktop and laptop computers are limited in the type of data that can be collected. For instance, even if laptops are suitable for mobile use, most do not have GPS sensors and are therefore not location aware. The aggregation of the collected data provides an opportunity to gain insight into mobile users' profiles and then personalize the mobile experience of a particular user.In recent years, deep learning has become the method of choice for solving complex representation learning problems. 1 The computational model comprises a set of processing layers, each of which successively transforms its input into a slightly more abstract representation than that in the previous layer. One fundamental aspect of deep learning is that each layer relies on a simple module that does not require any particular domain expertise for its design. This stack of processing layers attempts to produce a new set of input data that reduces irrelevant variations and amplifies discriminative information. Using a layer composition, deep learning methods can learn complex nonlinear functions by exploiting a general-purpose learning procedure. Deep learning is now regarded as the state-of-the-art method for solving many complex problems, such as image processing.The authors claim that deep le...

Accelerating On-Device Learning with Layer-Wise Processor Selection Method on Unified Memory

“…From a fundamental point of view, methods making efficient use of the shared memory of mobile devices have been proposed [ 22 , 23 ]. These studies used shared memory to eliminate the data copy time between the CPU and GPU [ 22 ] or prevent data duplication for the GPU [ 23 ]. However, it can only prevent the duplication of memory or eliminate the data copy time.…”

“…The implementation of deep learning training is more complex than that of deep learning inference owing to a lack of resources and the complexity of the process. To solve this issue, a study on deep learning training on mobile devices (DeepMobile) has been conducted [ 23 , 24 , 27 ]. DeepMobile [ 23 , 27 ] utilized shared memory to solve the memory shortage during training and to optimize mobile GPUs to accelerate training on mobile devices.…”

“…To solve this issue, a study on deep learning training on mobile devices (DeepMobile) has been conducted [ 23 , 24 , 27 ]. DeepMobile [ 23 , 27 ] utilized shared memory to solve the memory shortage during training and to optimize mobile GPUs to accelerate training on mobile devices. Another study profiled the latency, data copy time, and search processor pathing using dynamic programming without shared memory [ 24 ].…”

“…Therefore, existing studies applying deep learning algorithms to mobile devices mainly focus on accelerating the deep learning inference, which requires relatively low computing power. To accelerate the deep learning inference, existing approaches optimize inference for mobile processors [ 18 , 19 , 20 ] or perform inference by dividing the model across multiple computing resources of the mobile device [ 18 , 21 , 22 , 23 , 24 , 25 ]. Other approaches focus on enhancing the usability of memory to eliminate data copy time [ 22 , 23 ].…”

“…To accelerate the deep learning inference, existing approaches optimize inference for mobile processors [ 18 , 19 , 20 ] or perform inference by dividing the model across multiple computing resources of the mobile device [ 18 , 21 , 22 , 23 , 24 , 25 ]. Other approaches focus on enhancing the usability of memory to eliminate data copy time [ 22 , 23 ]. However, the existing approaches for accelerating deep learning inference cannot directly be applied for accelerating deep learning training because training the network model involves more complex components.…”