Dynamic Split Computing for Efficient Deep Edge Intelligence

Abstract

Deploying deep neural networks (DNNs) on IoT and mobile devices is a challenging task due to their limited computational resources. Thus, demanding tasks are often entirely offloaded to edge servers which can accelerate inference, however, it also causes communication cost and evokes privacy concerns. In addition, this approach leaves the computational capacity of end devices unused. Split computing is a paradigm where a DNN is split into two sections; the first section is executed on the end device, and the output is transmitted to the edge server where the final section is executed. Here, we introduce dynamic split computing, where the optimal split location is dynamically selected based on the state of the communication channel. By using natural bottlenecks that already exist in modern DNN architectures, dynamic split computing avoids retraining and hyperparameter optimization, and does not have any negative impact on the final accuracy of DNNs. Through extensive experiments, we show that dynamic split computing achieves faster inference in edge computing environments where the data rate and server load vary over time.

Efficient High-Resolution Deep Learning: A Survey

“…For instance, the downsampled images in NUD and compressed representation in TOIC can be transmitted instead of the original inputs. This would be a form of split computing (also known as collaborative intelligence) [166], [167], where the initial portion of computation is performed on a resource-constrained end-device, and the compact intermediate representation is then transmitted to a server where the rest of the computation is carried out. A study using this idea for high-resolution images captured by drones is reported in [168].…”

Edge device for movement pattern classification using neural network algorithms

“…In this context, solutions can be created to bring computational capacity closer to the edge of the network to avoid data transmission to the cloud for processing and alleviate critical problems related to latency [14], energy consumption [15], bandwidth and scalability [16]. Furthermore, the integration with artificial intelligence empowers machines with human-like intelligence and includes knowledge-based perception and decision-making capabilities [17]. Therefore, in this research we focus on evaluating algorithms based on neural networks adapted to an edge device in the context of a human activity recognition application [18].…”