TeamNet: A Collaborative Inference Framework on the Edge

Fang, Yihao; Jin, Ziyi; Zheng, Rong

doi:10.1109/icdcs.2019.00148

Cited by 8 publications

(7 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…TeamNet [6] takes a different approach for computation partitioning. Rather than dividing a pre-trained neural network structurally, it explores knowledge specialization and trains multiple small NNs through competitive and selective learning.…”

Section: Computation and Model Partitioningmentioning

confidence: 99%

“…Instead, only one submodel is applied. This is one of the key differences between CacheNet and the work in [6]. S-InfoVAE can be executed on the end device or on the edge server.…”

Section: Submodel Selectionmentioning

confidence: 99%

“…The challenges of partitioning are two folds: 1) each partition must be sufficiently specialized and the combination (collaboration) of all partitions must behave roughly equivalently to the original neural network; 2) There must be a selector that picks the right partition given a specific hint at a time. The first challenge was mostly solved by TeamNet [6], while the second one has not been solved by any approaches at this point.…”

Section: System Designmentioning

confidence: 99%

“…Thus, in training (Figure 1a), we need to 1) maximize the mutual information between the code representation and the input image; 2) better associate the code representation with the specialized neural network partition; 3) train each partition with respect to their output entropy, which has been demonstrated practical in TeamNet [6]. CacheNet's system design is therefore conducted simultaneously with respect to the above objectives.…”

Section: System Designmentioning

confidence: 99%

“…This question has to some degree been answered by partitioning approaches [6], [12], [20]. They mainly fall into two paradigms: 1) model partitioning: concurrent computing among edge nodes and/or end devices [6], which collaboratively performs inference in parallel per a particular sensor input; 2) computation partitioning: partition between edge and cloud, which take a pre-trained deep model and decide at run-time based on computation capability of local and cloud compute nodes and communication overheads where portions of computation should reside [12]. The inference time of both paradigms is clearly lower bounded by the smaller (or smallest) of inference times on the end device and a cloud node (or on all end devices/edge nodes).…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

CacheNet: A Model Caching Framework for Deep Learning Inference on the Edge

Fang¹,

Shalmani²,

Zheng³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

The success of deep neural networks (DNN) in machine perception applications such as image classification and speech recognition comes at the cost of high computation and storage complexity. Inference of uncompressed large scale DNN models can only run in the cloud with extra communication latency back and forth between cloud and end devices, while compressed DNN models achieve real-time inference on end devices at the price of lower predictive accuracy. In order to have the best of both worlds (latency and accuracy), we propose CacheNet, a model caching framework. CacheNet caches low-complexity models on end devices and high-complexity (or full) models on edge or cloud servers. By exploiting temporal locality in streaming data, high cache hit and consequently shorter latency can be achieved with no or only marginal decrease in prediction accuracy. Experiments on CIFAR-10 and FVG have shown CacheNet is 58 − 217% faster than baseline approaches that run inference tasks on end devices or edge servers alone.

show abstract

Section: Computation and Model Partitioningmentioning

confidence: 99%

“…Instead, only one submodel is applied. This is one of the key differences between CacheNet and the work in [6]. S-InfoVAE can be executed on the end device or on the edge server.…”

Section: Submodel Selectionmentioning

confidence: 99%

Section: System Designmentioning

confidence: 99%

Section: System Designmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

CacheNet: A Model Caching Framework for Deep Learning Inference on the Edge

Fang¹,

Shalmani²,

Zheng³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

StAIn: Stealthy Avenues of Attacks on Horizontally Collaborated Convolutional Neural Network Inference and Their Mitigation

et al. 2023

View full text Add to dashboard Cite

With significant potential improvement in device-to-device (D2D) communication due to improved wireless link capacity (e.g., 5G and NextG systems), a collaboration of multiple edge devices (called horizontal collaboration (HC)) is becoming a reality for real-time Edge Intelligence (EI). The distributed nature of HC offers an advantage against traditional adversarial attacks because the adversary does not have access to the entire deep learning architecture (DLA). Due to the involvement of multiple untrusted edge devices in HC environment, the possibility of malicious devices cannot be eliminated. In this paper, we unearth some attacks that are very effective and stealthy even when the attacker has minimal knowledge of the DLA as is the case in HC-based DLA. We are also providing novel filtering methods to mitigate such attacks. Our novel attacks leverage local information available on output feature maps (FMs) of a targeted edge device to modify the regular adversarial attacks (e.g. Fast Gradient Signed Method (FGSM) and Jacobian-based Saliency Map Attack (JSMA)). Similarly, a customized convolutional neural network (CNN) based filter is empirically designed, developed, and tested. Four different CNN models (LeNet, CapsuleNet, MiniVGGNet, and VGG16) are used to validate the proposed attacks and defense methodologies. Our three attacks on four different CNN models (with two variations of each attack) show a substantial accuracy drop of 62% on average. The proposed filtering approach is able to mitigate the attack by recovering the actual accuracy back to 75.1% on average. To the best of our knowledge, this is the first work that investigates the security vulnerability of DLA in the HC environment, and all three of our attacks are scalable and agnostic to the partition location within the DLA.

show abstract