JointDNN: An Efficient Training and Inference Engine for Intelligent Mobile Cloud Computing Services

Eshratifar, Amir Erfan; Abrishami, Mohammad Saeed; Pedram, Massoud

doi:10.48550/arxiv.1801.08618

Cited by 8 publications

(36 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, when training the DNN with the back-propagation rule in the stochastic gradient descent (SGD) algorithm [12], the workers need to communicate to exchange the intermediate results. The works of JointDNN [8] and JALAD [13] demonstrate the effectiveness of the model parallelism method. However, since the layers of the DNN are trained sequentially, when one worker is computing the others must stay idle.…”

Section: Background and Motivationmentioning

confidence: 89%

“…2) All-Cloud: The edge device transmits all the training data samples to the cloud center, and the cloud center completes the DNN training. 3) JointDNN [8]: The edge device and the cloud center jointly train the DNNs.…”

Section: Baselinesmentioning

confidence: 99%

“…[27] presented an optimal scheduling algorithm for collaboratively computation of feedforward neural networks to achieve maximum performance and energy efficiency. JointDNN [8] provided optimization formulations at layer-granularity for forward and backward propagation in DNNs, which can get the optimal computation scheduling of processing some layers on the edge device and some layers on the cloud server. The limitation of cloud-based approach is that the long WAN latency between device and cloud.…”

Section: A Cloud-basedmentioning

confidence: 99%

“…There are also hierarchical training approaches to efficient training of DNNs. JointDNN is proposed in [8], which trains some layers of a DNN on an edge device and the other layers on the cloud center. However, the latency between the edge device and the cloud center is still the major factor to limit the training speed.…”

Section: Introductionmentioning

confidence: 99%

“…1 illustrates the difference between the horizontal training and hierarchical training paradigms. Observing that the works in [8] and [9] only consider two levels in the mobile-edge-cloud hierarchical architecture -the device and cloud levels in [8] and the mobile and edge levels in [9], the drawback of them is that they did not fully utilize the communication and computation resources of all the three levels. As communication latency between mobile and edge levels is generally low and the computation resource at the cloud level is abundant, a holistic framework that fully exploits the communication and computation resources of all the three levels can definitely leash the great potentials of mobile-edgecloud computing for accelerating edge AI learning.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

HierTrain: Fast Hierarchical Edge AI Learning with Hybrid Parallelism in Mobile-Edge-Cloud Computing

Liu¹,

Chen²,

Zhou³

et al. 2020

Preprint

View full text Add to dashboard Cite

Nowadays, deep neural networks (DNNs) are the core enablers for many emerging edge AI applications. Conventional approaches to training DNNs are generally implemented at central servers or cloud centers for centralized learning, which is typically timeconsuming and resource-demanding due to the transmission of a large amount of data samples from the device to the remote cloud. To overcome these disadvantages, we consider accelerating the learning process of DNNs on the Mobile-Edge-Cloud Computing (MECC) paradigm. In this paper, we propose HierTrain, a hierarchical edge AI learning framework, which efficiently deploys the DNN training task over the hierarchical MECC architecture. We develop a novel hybrid parallelism method, which is the key to HierTrain, to adaptively assign the DNN model layers and the data samples across the three levels of edge device, edge server and cloud center. We then formulate the problem of scheduling the DNN training tasks at both layer-granularity and sample-granularity. Solving this optimization problem enables us to achieve the minimum training time. We further implement a hardware prototype consisting of an edge device, an edge server and a cloud server, and conduct extensive experiments on it. Experimental results demonstrate that HierTrain can achieve up to 6.9× speedup compared to the cloud-based hierarchical training approach.

show abstract

Section: Background and Motivationmentioning

confidence: 89%

Section: Baselinesmentioning

confidence: 99%

Section: A Cloud-basedmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

HierTrain: Fast Hierarchical Edge AI Learning with Hybrid Parallelism in Mobile-Edge-Cloud Computing

Liu¹,

Chen²,

Zhou³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

Back-And-Forth Prediction for Deep Tensor Compression

Choi

Cohen

Bajić

2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Recent AI applications such as Collaborative Intelligence with neural networks involve transferring deep feature tensors between various computing devices. This necessitates tensor compression in order to optimize the usage of bandwidthconstrained channels between devices. In this paper we present a prediction scheme called Back-and-Forth (BaF) prediction, developed for deep feature tensors, which allows us to dramatically reduce tensor size and improve its compressibility. Our experiments with a state-of-the-art object detector demonstrate that the proposed method allows us to significantly reduce the number of bits needed for compressing feature tensors extracted from deep within the model, with negligible degradation of the detection performance and without requiring any retraining of the network weights. We achieve a 62% and 75% reduction in tensor size while keeping the loss in accuracy of the network to less than 1% and 2%, respectively.

show abstract

Deep Feature Compression for Collaborative Object Detection

Choi

Bajić

2018

2018 25th IEEE International Conference on Image Processing (ICIP)

126

103

View full text Add to dashboard Cite

Recent studies have shown that the efficiency of deep neural networks in mobile applications can be significantly improved by distributing the computational workload between the mobile device and the cloud. This paradigm, termed collaborative intelligence, involves communicating feature data between the mobile and the cloud. The efficiency of such approach can be further improved by lossy compression of feature data, which has not been examined to date. In this work we focus on collaborative object detection and study the impact of both near-lossless and lossy compression of feature data on its accuracy. We also propose a strategy for improving the accuracy under lossy feature compression. Experiments indicate that using this strategy, the communication overhead can be reduced by up to 70% without sacrificing accuracy.

show abstract

JointDNN: An Efficient Training and Inference Engine for Intelligent Mobile Cloud Computing Services

Cited by 8 publications

References 0 publications

HierTrain: Fast Hierarchical Edge AI Learning with Hybrid Parallelism in Mobile-Edge-Cloud Computing

HierTrain: Fast Hierarchical Edge AI Learning with Hybrid Parallelism in Mobile-Edge-Cloud Computing

Back-And-Forth Prediction for Deep Tensor Compression

Deep Feature Compression for Collaborative Object Detection

Contact Info

Product

Resources

About