Moving Deep Learning to the Edge

Abstract

Deep learning is now present in a wide range of services and applications, replacing and complementing other machine learning algorithms. Performing training and inference of deep neural networks using the cloud computing model is not viable for applications where low latency is required. Furthermore, the rapid proliferation of the Internet of Things will generate a large volume of data to be processed, which will soon overload the capacity of cloud servers. One solution is to process the data at the edge devices themselves, in order to alleviate cloud server workloads and improve latency. However, edge devices are less powerful than cloud servers, and many are subject to energy constraints. Hence, new resource and energy-oriented deep learning models are required, as well as new computing platforms. This paper reviews the main research directions for edge computing deep learning algorithms.

Knowledge distillation in deep learning and its applications

“…Moreover, unpredictable network connections between the cloud and the device can also pose significant challenges. Thus, running the deep learning system on local devices is an important requirement in many domains and has a wide variety of applications including smart cities, self-driving cars, smart homes, medical devices and entertainment Véstias et al (2020) . Knowledge distillation allows developers to shrink down the size of deep learning models in order for them to fit into resource-limited devices having limited memory and power as illustrated in Fig.…”

A Classification of the Enabling Techniques for Low Latency and Reliable Communications in 5G and Beyond: AI-Enabled Edge Caching

“…The increased popularity of DL in edge caching and wireless networks is mainly due to the following reasons:  There is a huge amount of data traffic generated on the Internet. For instance, rapid advancement of IoT and social networking applications results in an exponential growth of the data traffic generated at the network edge [34], [79], [91], [137].  The technique of DL has the ability to extract accurate information from raw data obtained from devices such as IoT devices which are deployed in complex environments [130].…”

“…However, traditional DL schemes are cloudcentric and they require a stream of raw training data to be sent and processed in a centralized server [39], [74]. The process of sending streams of raw training data to a centralized server can result in several challenges, including slow response to real-time events in latency sensitive applications, excessive network communication resource consumption, increased network traffic, high energy consumption, and reduced privacy of training data [39], [43], [79], [87], [124], [137], [138]. Therefore, traditional DL frameworks may not be suitable in application scenarios with large-scale data that require low latency, efficiency, and scalability [87], [137].…”

“…The process of sending streams of raw training data to a centralized server can result in several challenges, including slow response to real-time events in latency sensitive applications, excessive network communication resource consumption, increased network traffic, high energy consumption, and reduced privacy of training data [39], [43], [79], [87], [124], [137], [138]. Therefore, traditional DL frameworks may not be suitable in application scenarios with large-scale data that require low latency, efficiency, and scalability [87], [137]. Furthermore, it may be impractical to employ traditional DL frameworks in applications with privacy sensitive training data [39], [87].…”

Probabilistic Predictions with Federated Learning

“…The data are distributed over a large group of end devices and are commonly transferred to a central server, where the learning of the ML models can be performed using the entire dataset. This poses two problems: transferring the data may lead to high communication costs and the privacy of the users may be compromised [ 1 ]. To counter these problems, ML can be performed on-device, so that the data are kept localized on the device and are not uploaded to a central server.…”