DeepSplit: Dynamic Splitting of Collaborative Edge-Cloud Convolutional Neural Networks

Mehta, Rishabh; Shorey, Rajeev

doi:10.1109/comsnets48256.2020.9027432

Cited by 21 publications

(14 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several approaches have been proposed in the literature for improving the performance of inference in cloud-edge continuum [33][34][35][36][37]. The main difference is that we do distribute inference between nodes in cloud-edge continuum, due to the argued benefits of our approach (Table 1).…”

Section: Discussionmentioning

confidence: 99%

“…Additionally, the trained neural network (NN) may be partitioned between the cloud-edge continuum for reducing latency of ML inference [33]. In the partitioning, the impact of different Convolutional Neural Network (CNN)-layers on memory consumption, computation, and bandwidth requirements (when transferring the layers) should be understood [34,35]. Trained models may also be distributed among a set of edge-nodes for reducing latency of inference [36].…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Architecture for Enabling Edge Inference via Model Transfer from Cloud Domain in a Kubernetes Environment

et al. 2020

View full text Add to dashboard Cite

The current approaches for energy consumption optimisation in buildings are mainly reactive or focus on scheduling of daily/weekly operation modes in heating. Machine Learning (ML)-based advanced control methods have been demonstrated to improve energy efficiency when compared to these traditional methods. However, placing of ML-based models close to the buildings is not straightforward. Firstly, edge-devices typically have lower capabilities in terms of processing power, memory, and storage, which may limit execution of ML-based inference at the edge. Secondly, associated building information should be kept private. Thirdly, network access may be limited for serving a large number of edge devices. The contribution of this paper is an architecture, which enables training of ML-based models for energy consumption prediction in private cloud domain, and transfer of the models to edge nodes for prediction in Kubernetes environment. Additionally, predictors at the edge nodes can be automatically updated without interrupting operation. Performance results with sensor-based devices (Raspberry Pi 4 and Jetson Nano) indicated that a satisfactory prediction latency (~7–9 s) can be achieved within the research context. However, model switching led to an increase in prediction latency (~9–13 s). Partial evaluation of a Reference Architecture for edge computing systems, which was used as a starting point for architecture design, may be considered as an additional contribution of the paper.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Architecture for Enabling Edge Inference via Model Transfer from Cloud Domain in a Kubernetes Environment

et al. 2020

View full text Add to dashboard Cite

show abstract

“…In tasks such as image recognition, there is no straightforward implementation. However, several strategies have been studied for optimal splitting of CNNs on edge devices [13], [30]- [33]. Other techniques have studied the performance of the above approaches on cloud and mobile devices separately [34].…”

Section: Related Workmentioning

confidence: 99%

“…Another popular approach is splitting the CNN between the smartphone and a cloud server where a part of the processing is done on the cloud server. Such splitting could be done using N-step algorithms [13] or using latency-based optimisation [14]. These approaches have three major shortcomings when applied to smartphones.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

SmartSplit: Latency-Energy-Memory Optimisation for CNN Splitting on Smartphone Environment

Prakash,

Bansal,

Verma

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Artificial Intelligence has now taken centre stage in the smartphone industry owing to the need of bringing all processing close to the user and addressing privacy concerns. Convolution Neural Networks (CNNs), which are used by several AI applications, are highly resource and computation intensive. Although new generation smartphones come with AI-enabled chips, minimal memory and energy utilisation is essential as many applications are run concurrently on a smartphone. In light of this, optimising the workload on the smartphone by offloading a part of the processing to a cloud server is an important direction of research. In this paper, we analyse the feasibility of splitting CNNs between smartphones and cloud server by formulating a multi-objective optimisation problem that optimises the end-to-end latency, memory utilisation, and energy consumption. We design SmartSplit, a Genetic Algorithm with decision analysis based approach to solve the optimisation problem. Our experiments run with multiple CNN models show that splitting a CNN between a smartphone and a cloud server is feasible. The proposed approach, SmartSplit fares better when compared to other state-of-the-art approaches.

show abstract

Hybrid Partitioning for Embedded and Distributed CNNs Inference on Edge Devices

Kaboubi

Letondeur

Coupaye

et al. 2023

Communications in Computer and Information Science

View full text Add to dashboard Cite

Convolutional Neural Networks (CNNs) and Deep Neural Networks (DNNs) are ubiquitously utilized in many Internet of Things applications, especially for real-time image-based analysis. In order to cope with concerns such as resiliency, privacy and near real time analysis, these models must be deployed on edge devices. Particularly for large models, the large number of parameters becomes a bottleneck for the inference process because edge devices are resource constrained, subjects to failures and/or hardware faults. New solutions to cope with these issues are required. This paper proposes a hybrid partitioning strategy, architecture and implementation (called HyPS), which identifies the best positions in the model structure to split the network structure into small partitions that fit resources constraints of edge devices noticeably by decreasing instantaneous memory needs. The generated partitions consume less memory than the original network and each partition can be processed almost separately, resulting in new ways to process CNN's execution at the edge. Thanks to this partitioning strategy, large CNNs inference can be run without modifying the main model architecture. The proposed approach is assessed on the well-known neural network structure of VGG16 for image classification. The results of the experimental campaign show that the partitioning method allows for the successful inference of large models on devices with limited overhead and high accuracy.

show abstract

DeepSplit: Dynamic Splitting of Collaborative Edge-Cloud Convolutional Neural Networks

Cited by 21 publications

References 7 publications

Architecture for Enabling Edge Inference via Model Transfer from Cloud Domain in a Kubernetes Environment

Architecture for Enabling Edge Inference via Model Transfer from Cloud Domain in a Kubernetes Environment

SmartSplit: Latency-Energy-Memory Optimisation for CNN Splitting on Smartphone Environment

Hybrid Partitioning for Embedded and Distributed CNNs Inference on Edge Devices

Contact Info

Product

Resources

About