Distilled Split Deep Neural Networks for Edge-Assisted Real-Time Systems

Matsubara, Yoshitomo; Baidya, Sabur; Callegaro, Davide; Levorato, Marco; Singh, Sameer

doi:10.1145/3349614.3356022

Cited by 79 publications

(81 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…If the prediction results are to be sent back to the mobile device, a further communication delay term should be taken into account, although outcomes (e.g., bounding boxes and labels) typically have a much smaller size compared to the input image. As discussed in [10,16], the delay of the communication from mobile device to edge computer is a critical component of the total inference time, which may become dominant in some network conditions, where the performance of edge computing may suffer from a reduced channel capacity.…”

Section: Mobile and Edge Computingmentioning

confidence: 99%

“…Building on the work of Kang et al [10], recent contributions propose DNN splitting methods [2,3,8,11,16,24,26]. Most of these studies, however, (I) do not evaluate models using their proposed lossy compression techniques [2], (II) lack of motivation to split the models as the size of the input data is exceedingly small, e.g., 32 × 32 pixels RGB images in [8,24,26], (III) specifically select models and network conditions in which their proposed method is advantageous [11], and/or (IV) assess proposed models in simple classification tasks such as miniImageNet, Caltech 101, CIFAR -10, and -100 datasets [3,8,16,24].…”

Section: Split Computingmentioning

confidence: 99%

“…Recently, an intermediate option, namely split Deep Neural Network (DNN) or split computing, has been attracting a considerable interest [2,3,8,10,11,16,24,26]. Many of such methods literally split DNN models into head and tail portions, which are executed by the mobile device and edge computer, respectively.…”

Section: Introductionmentioning

confidence: 99%

“…Note that in this case, instead of the input data, the tensor produced by the head model should be transported to the edge computer. More sophisticated approaches [3,8,16,24] modify the architecture of the model itself to (a) reduce the size of the tensor to be transferred, and (b) reduce the computing load assigned to the mobile device. Intuitively, the two features mentioned above would prove essential in enabling effective task offloading in challenged settings (e.g., low capacity of the wireless channel and low computing capacity at the mobile device).…”

Section: Introductionmentioning

confidence: 99%

“…Although promising, as discussed in detail in Section 3.2, most of the existing split DNN approaches are either not evaluated [2] or evaluated only in simple classification tasks [3,8,16,24,26], such as on miniImageNet, Caltech 101, CIFAR -10, and -100 datasets. The goal of this paper is to discuss the several technical challenges in achieving effective DNN splitting for edge computing in one of the most difficult computing tasks, that is, object detection.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Split Computing for Complex Object Detectors

Matsubara

Levorato

2020

Proceedings of the 4th International Workshop on Embedded and Mobile Deep Learning

Self Cite

View full text Add to dashboard Cite

Following the trends of mobile and edge computing for DNN models, an intermediate option, split computing, has been attracting attentions from the research community. Previous studies empirically showed that while mobile and edge computing often would be the best options in terms of total inference time, there are some scenarios where split computing methods can achieve shorter inference time. All the proposed split computing approaches, however, focus on image classification tasks, and most are assessed with small datasets that are far from the practical scenarios. In this paper, we discuss the challenges in developing split computing methods for powerful R-CNN object detectors trained on a large dataset, COCO 2017. We extensively analyze the object detectors in terms of layer-wise tensor size and model size, and show that naive split computing methods would not reduce inference time. To the best of our knowledge, this is the first study to inject small bottlenecks to such object detectors and unveil the potential of a split computing approach. CCS CONCEPTS • Computing methodologies → Computer vision; • Information systems → Computing platforms.

show abstract

Section: Mobile and Edge Computingmentioning

confidence: 99%

Section: Split Computingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Split Computing for Complex Object Detectors

Matsubara

Levorato

2020

Proceedings of the 4th International Workshop on Embedded and Mobile Deep Learning

Self Cite

View full text Add to dashboard Cite

show abstract

torchdistill: A Modular, Configuration-Driven Framework for Knowledge Distillation

Matsubara¹

2021

Reproducible Research in Pattern Recognition

Self Cite

View full text Add to dashboard Cite

Reproducibility in scientific work has been becoming increasingly important in research communities such as machine learning, natural language processing, and computer vision communities due to the rapid development of the research domains supported by recent advances in deep learning. In this work, we present a significantly upgraded version of torchdistill 1 , a modular-driven coding-free deep learning framework significantly upgraded from the initial release, which supports only image classification and object detection tasks for reproducible knowledge distillation experiments. To demonstrate that the upgraded framework can support more tasks with third-party libraries, we reproduce the GLUE benchmark results of BERT models using a script based on the upgraded torchdistill, harmonizing with various Hugging Face libraries. All the 27 fine-tuned BERT models and configurations to reproduce the results are published at Hugging Face 2 , and the model weights have already been widely used in research communities. We also reimplement popular small-sized models and new knowledge distillation methods and perform additional experiments for computer vision tasks.

show abstract