Split Computing for Complex Object Detectors

Matsubara, Yoshitomo; Levorato, Marco

doi:10.1145/3410338.3412338

Cited by 22 publications

(23 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, splitting at late layers would position most of the computational complexity at the weaker mobile device. This issue was recently discussed in [3], [10] for image classification and object detection models, reinforcing the results obtained in [7] on traditional network splitting.…”

Section: A Backgroundsupporting

confidence: 69%

“…This paper builds on this approach [3], [10] to obtain innetwork compression with further improved detection performance in object detection tasks. Specifically, we generalize the head network distillation technique, and apply it to the state of the art detection models described in the previous section (Faster R-CNN, Mask R-CNN, and Keypoint R-CNN).…”

Section: A Backgroundmentioning

confidence: 99%

“…This makes the distillation of head models difficult, as they would need to embed multiple bottleneck layers at the points in the head network whose output is forwarded to the detectors. Clearly, the amount of data transmitted over the network would be inevitably larger as multiple outputs would need to be transferred [10].…”

Section: A Backgroundmentioning

confidence: 99%

“…We build on the results presented in [3], where we applied a similar concept to image classification tasks. The more convoluted structure of object detectors, where the output of intermediate layers is propagated to a final detection module, introduces further challenges discussed in [10] that we address by proposing a generalized loss function guiding the distillation process and applying a bottleneck quantization technique.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Neural Compression and Filtering for Edge-assisted Real-time Object Detection in Challenged Networks

Matsubara,

Levorato

2020

Preprint

Self Cite

View full text Add to dashboard Cite

The edge computing paradigm places computecapable devices -edge servers -at the network edge to assist mobile devices in executing data analysis tasks. Intuitively, offloading compute-intense tasks to edge servers can reduce their execution time. However, poor conditions of the wireless channel connecting the mobile devices to the edge servers may degrade the overall capture-to-output delay achieved by edge offloading. Herein, we focus on edge computing supporting remote object detection by means of Deep Neural Networks (DNNs), and develop a framework to reduce the amount of data transmitted over the wireless link. The core idea we propose builds on recent approaches splitting DNNs into sections -namely head and tail models -executed by the mobile device and edge server, respectively. The wireless link, then, is used to transport the output of the last layer of the head model to the edge server, instead of the DNN input. Most prior work focuses on classification tasks and leaves the DNN structure unaltered. Herein, our focus is on DNNs for three different object detection tasks, which present a much more convoluted structure, and modify the architecture of the network to: (i) achieve in-network compression by introducing a bottleneck layer in the early layers on the head model, and (ii) prefilter pictures that do not contain objects of interest using a convolutional neural network. Results show that the proposed technique represents an effective intermediate option between local and edge computing in a parameter region where these extreme point solutions fail to provide satisfactory performance. We release the code and trained models at https://github.com/ yoshitomo-matsubara/hnd-ghnd-object-detectors.

show abstract

Section: A Backgroundsupporting

confidence: 69%

Section: A Backgroundmentioning

confidence: 99%

Section: A Backgroundmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Neural Compression and Filtering for Edge-assisted Real-time Object Detection in Challenged Networks

Matsubara,

Levorato

2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…• We apply BottleFit on cutting-edge CNNs such as DenseNet-169, DenseNet-201 and ResNet-152 on the ImageNet dataset, and compare the accuracy obtained by BottleFit with state-of-the-art local computing [6] and split computing approaches [16]- [19], [21], [24]. Our training campaign concludes that BottleFit achieves up to 77.1% data compression (with respect to JPEG) with only up to 0.6% loss in accuracy, while existing mobile and split computing approaches incur considerable accuracy loss of up to 6% and 3.6%, respectively.…”

Section: Introductionmentioning

confidence: 99%

BottleFit: Learning Compressed Representations in Deep Neural Networks for Effective and Efficient Split Computing

Matsubara,

Callegaro,

Singh

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Although mission-critical applications require the use of deep neural networks (DNNs), their continuous execution at mobile devices results in a significant increase in energy consumption. While edge offloading can decrease energy consumption, erratic patterns in channel quality, network and edge server load can lead to severe disruption of the system's key operations. An alternative approach, called split computing, generates compressed representations within the model (called "bottlenecks"), to reduce bandwidth usage and energy consumption. Prior work has proposed approaches that introduce additional layers, to the detriment of energy consumption and latency. For this reason, we propose a new framework called BottleFit, which, in addition to targeted DNN architecture modifications, includes a novel training strategy to achieve high accuracy even with strong compression rates. We apply BottleFit on cutting-edge DNN models in image classification, and show that BottleFit achieves 77.1% data compression with up to 0.6% accuracy loss on ImageNet dataset, while state of the art such as SPINN loses up to 6% in accuracy. We experimentally measure the power consumption and latency of an image classification application running on an NVIDIA Jetson Nano board (GPU-based) and a Raspberry PI board (GPU-less). We show that BottleFit decreases power consumption and latency respectively by up to 49% and 89% with respect to (w.r.t.) local computing and by 37% and 55% w.r.t. edge offloading. We also compare BottleFit with state-of-the-art autoencoders-based approaches, and show that (i) BottleFit reduces power consumption and execution time respectively by up to 54% and 44% on the Jetson and 40% and 62% on Raspberry PI; (ii) the size of the head model executed on the mobile device is 83 times smaller. The code repository will be published for full reproducibility of the results.

show abstract

Boosting Cost‐Efficiency in Robotics: A Distributed Computing Approach for Harvesting Robots

Xie,

Li,

Feng

et al. 2024

Journal of Field Robotics

View full text Add to dashboard Cite

Multi‐arm harvesting robots offer a promising solution to the labor shortage in fruit harvesting, due to their ability to improve harvesting efficiency. However, multi‐arm harvesters necessitate additional visual sensors to acquire distribution information of fruits within larger working spaces. Greater demands are consequently imposed on graphics computation, leading to increased costs in computing hardware of robot system. To balance the graphics computing cost and reduce energy consumption, distributed graphics computation frameworks for multi‐arm robot vision system are proposed in this study. First, a host‐edge framework is proposed to assign the tasks of image inference and depth alignment to host computer and edge computing modules through a decentralized mode of local connection. Moreover, to increase the endurance time of robot in application, the edge computing modules are reduced and the fifth generation mobile communication is integrated into robot graphics computing system to transfer on‐board image processing to a remote computing server with MQTT protocol. To verify the effectiveness of the proposed framework, comprehensive experiments were performed, demonstrating that, compared with traditional computing framework, the proposed local distributed framework reduced 35.6% average time consumption, and over 20 FPS average processing speed can be achieve. The remote distributed framework has reduced the computational power consumption of the on‐board system by approximately 23.1% while ensuring the performance is not lower than the local distributed framework. Finally, by discussing the two frameworks in terms of stability and cost, we present the commercial viability for the application of multi‐arm harvesting robot.

show abstract

Split Computing for Complex Object Detectors

Cited by 22 publications

References 22 publications

Neural Compression and Filtering for Edge-assisted Real-time Object Detection in Challenged Networks

Neural Compression and Filtering for Edge-assisted Real-time Object Detection in Challenged Networks

BottleFit: Learning Compressed Representations in Deep Neural Networks for Effective and Efficient Split Computing

Boosting Cost‐Efficiency in Robotics: A Distributed Computing Approach for Harvesting Robots

Contact Info

Product

Resources

About