BottleNet: A Deep Learning Architecture for Intelligent Mobile Cloud Computing Services

Abstract

Recent studies have shown the latency and energy consumption of deep neural networks can be significantly improved by splitting the network between the mobile device and cloud. This paper introduces a new deep learning architecture, called BottleNet, for reducing the feature size needed to be sent to the cloud. Furthermore, we propose a training method for compensating for the potential accuracy loss due to the lossy compression of features before transmitting them to the cloud. BottleNet achieves on average 30× improvement in end-to-end latency and 40× improvement in mobile energy consumption compared to the cloud-only approach with negligible accuracy loss.

Back-And-Forth Prediction for Deep Tensor Compression

“…Recent studies [3][4][5][6] have discussed various methods to compress deep feature tensors transmitted to the cloud. In [3], a feature map quantized to 8 bits was compressed by PNG, which is a lossless compression tool, so that the degradation of inference accuracy was negligible.…”

“…However, this approach can cause congestion problems due to the growth of the volume of data transmitted over the network and the number of devices linked to the cloud. To address this problem, recent studies in collaborative intelligence have developed optimized deployment strategies [2][3][4][5][6][7].…”

“…Previous studies [4,5] have explored the efficacy of compressing deep feature tensors using conventional standard codecs, in the context of object detection and image classification. Also, [6] suggests a method to first reduce the dimensionality of the deep feature tensor, then compress it using a codec.…”

Bit Allocation for Multi-Task Collaborative Intelligence

“…From (27) and (28) we conclude that there is at least one i such that D i (r ) > D i (r ), thereby contradicting (26). Hence, r is Pareto-optimal.…”

“…so r = r min is the point that satisfies (26). On the other hand, if r > r max , then r is "to the right" of all distortion minima, so by the strict convexity of D i (r) we have ∀i, min…”